Server and Non-Transitory Computer Readable Medium Storing Program For Remote Conference

ABSTRACT

A server obtains video data and layout information. The layout information includes at least a portion of identification information and status information associated with each other. The status information relates to a display condition of each of a plurality of captured images in a video display region. The server judges whether a specific captured image is included in the video display region. The server determines update frequency information when the specific captured image is not included in the video display region. The update frequency information sets a portion of a plurality of inter-frame coded image data included in the video data of the second communication device as a non-target for transmission. The server transmits the video data in which the portion of the plurality of inter-frame coded image data are not included.

CROSS-REFERENCES TO RELATED APPLICATION

This application claims priority to Japanese Patent Application No.2013-270727 filed on Dec. 27, 2013, the content of which is hereinincorporated by reference in its entirety.

TECHNICAL FIELD

Aspects relate to a server, a method, and a non-transitory storagemedium storing programs, for performing a remote conference using aplurality of communication devices via a network.

BACKGROUND

There has been known a technique relating to a remote conference. Forexample, a known communication device is installed at each location fora television conference in a television conference system. In thetelevision conference system, each communication device is connected tothe other communication devices to perform a television conference. Eachcommunication device includes a monitor, on which a plurality of windowsis displayed. Upon receipt of moving image data from one or more othercommunication devices installed at other respective locations, acommunication device displays moving images of the respective locationsin respective windows on the monitor. In the communication device, forexample, when a size of a window in which a moving image is displayed isreduced, a control command for decreasing a frame rate or resolution ofmoving image data is generated. For another example, when a window isdismissed, a control command for stopping transmission of moving imagedata is generated. Such a control command is transmitted to acommunication device that sent the moving image data.

SUMMARY

A remote conference may be performed using a plurality of communicationdevices connected to a network. Data of video and audio in a remoteconference may be communicated between the plurality of thecommunication devices via the server connected to the network. Eachcommunication device may display captured images corresponding to aplurality of pieces of video data transmitted from the othercommunication devices. The number of captured images to be displayed ineach communication device may increase with an increase in number ofcommunication devices to be used (e.g., locations) in a remoteconference. In some cases, the captured images of all the locationsmight not be displayed on a monitor at the same time in a communicationdevice. In order to display one or more hidden captured images on themonitor, a user of the communication device may input, into thecommunication device, an instruction to move one or more capturedimages, through an operation for changing one or more captured images tobe displayed (e.g., scrolling, swiping, and/or changing a tab). Inresponse to the instruction, the communication device may display theone or more hidden captured images. When the communication devicereceives such an instruction while receipt of video data correspondingto the one or more hidden captured images has been stopped, thecommunication device starts receiving the corresponding video data inresponse to the instruction. Video data includes an intra-frame codedimage data and an inter-frame coded image data. The intra-frame codedimage data may be decoded without reference to information of anotherframe. In contrast to this, for example, an inter-frame coded image datamay be decoded with reference to information of an intra-frame codedimage data. Therefore, if the communication device receives aninter-frame coded image data in response to the moving instruction, thecommunication device might not decode video data until the communicationdevice receives an intra-frame coded image data. Owing to this, acaptured image corresponding to the video data might not be displayeduntil the communication device receives an intra-frame coded image data,whereby a hidden captured image might not be displayed smoothly inresponse to the changing operation in some cases.

Some embodiments provide for a server and a non-transitory storagemedium storing programs for a remote conference, which may enable one ormore hidden captured images to appear smoothly in a particularcommunication device while restricting increase of an amount of videodata to be transmitted to the communication device.

An aspect of the present disclosure is a server. The server obtains aplurality of video data and layout information. The layout informationincludes at least a portion of a plurality of identification informationand a plurality of status information associated with each other. Eachof the status information relates to a display condition of each of aplurality of captured images in a video display region. The serverjudges whether a specific captured image is included in the videodisplay region. The server determines update frequency information whenthe specific captured image is not included in the video display region.The update frequency information sets a portion of a plurality ofinter-frame coded image data included in the video data of the secondcommunication device as non-target for transmission. The servertransmits the video data in which the portion of the plurality ofinter-frame coded image data are not included.

DESCRIPTION OF THE DRAWINGS

Aspects of the disclosure are illustrated by way of example and not bylimitation in the accompanying figures in which like referencecharacters indicate similar elements.

FIG. 1 illustrates an example remote conference system in anillustrative embodiment according to one or more aspects of thedisclosure.

FIG. 2 illustrates a relationship between a video layout region in whichone or more captured images are laid out and a video display region of adisplay in the illustrative embodiment according to one or more aspectsof the disclosure.

FIG. 3 illustrates an example status table in the illustrativeembodiment according to one or more aspects of the disclosure.

FIG. 4 is a flowchart depicting example audio data transmittingprocessing in the illustrative embodiment according to one or moreaspects of the disclosure.

FIG. 5 is a flowchart depicting example video data transmittingprocessing in the illustrative embodiment according to one or moreaspects of the disclosure.

FIG. 6 is a flowchart depicting an example receiving processing in theillustrative embodiment according to one or more aspects of thedisclosure.

FIG. 7 is a flowchart depicting example layout information transmittingprocessing in the illustrative embodiment according to one or moreaspects of the disclosure.

FIG. 8 is a flowchart depicting example transferring processing in theillustrative embodiment according to one or more aspects of thedisclosure.

FIG. 9 is a continuation of the flowchart depicting the exampletransferring processing of FIG. 8 in the illustrative embodimentaccording to one or more aspects of the disclosure.

FIG. 10 is a continuation of the flowchart depicting the exampletransferring processing of FIG. 9 in the illustrative embodimentaccording to one or more aspects of the disclosure.

FIG. 11 illustrates example layout information stored in the server inthe illustrative embodiment according to one or more aspects of thedisclosure.

FIG. 12 illustrates an example update frequency setting stored in theserver in the illustrative embodiment according to one or more aspectsof the disclosure.

FIG. 13 is an explanatory diagram for explaining an update rule in theillustrative embodiment according to one or more aspects of thedisclosure.

FIG. 14 illustrates an example total audio play time table in theillustrative embodiment according to one or more aspects of thedisclosure.

FIG. 15 is a flowchart depicting example update frequency determiningprocessing in the illustrative embodiment according to one or moreaspects of the disclosure.

FIG. 16 is a flowchart depicting example transfer determining processingin the illustrative embodiment according to one or more aspects of thedisclosure.

DETAILED DESCRIPTION

Hereinafter, an illustrative embodiment for implementing one or moreaspects of the disclosure will be described with reference to theaccompanying drawings. The disclosure is not limited to specificembodiments, but various aspects may be adopted in the same technicalidea. For example, one or more aspects of the disclosure may be omittedor replaced with another one. The disclosure may include another aspectas well as the disclosure.

<Remote Conference System>

A remote conference system 10 will be described referring to FIGS. 1, 2,and 3. As depicted in FIG. 1, the remote conference system 10 includes aserver 20 and a plurality of, for example, six communication devices 30,71, 72, 73, 74, and 75. Hereinafter, a remote conference performed amongparties using the communication devices 30, 71, 72, 73, 74, and 75,respectively, will be described. In other embodiments, for example, aremote conference using the remote conference system 10 may be performedamong parties using five or less communication devices or using seven ormore communication devices.

The server 20 and the communication devices 30, 71, 72, 73, 74, and 75are connected to a network 90. The network 90 may be, for example, theInternet. A remote conference using the communication devices 30, 71,72, 73, 74, and 75 may be performed via the server 20 similar to a Webconference using a known remote conference system.

The communication device 30 has a function of performing communicationvia the network 90. The communication device 30 may be, for example, asmartphone, a tablet terminal, or a personal computer. In thisillustrative embodiment, the communication device 30 may be a mobiledevice, such as a smartphone or a tablet terminal. A detailedconfiguration of the communication device 30 will be described later.The communication devices 71, 72, 73, 74, and 75 may be a knowncommunication device. For example, one or more or all of thecommunication devices 71, 72, 73, 74, and 75 may be a communicationdevice that is the same as the communication device 30. In otherembodiments, for example, the communication devices 71, 72, 73, 74, and75 may be a personal computer. At a remote conference, the communicationdevices 71, 72, 73, 74, and 75 are configured to execute each processingthat the communication device 30 executes (refer to FIGS. 4, 7, 8, 9,and 10). In the illustrative embodiment, a description will be made bytaking the communication device 30 as an example.

In the remote conference system 10, for example, the communicationdevice 30 transmits video data and audio data to the server 20. Thevideo data transmitted from the communication device 30 corresponds toan image captured using the communication device 30 (hereinafter,referred to as “captured image”). The audio data transmitted from thecommunication device 30 corresponds to audio collected using thecommunication device 30. The communication device 71 also transmitsvideo data and audio data to the server 20. The video data transmittedfrom the communication device 71 corresponds to an image captured usingthe communication device 71. The audio data transmitted from thecommunication device 71 corresponds to audio collected using thecommunication device 71. Similar to the communication device 71, each ofthe communication devices 72, 73, 74, and 75 transmit, to the server 20,video data corresponding to an image captured using each of thecommunication devices 72, 73, 74, and 75 and audio data corresponding toaudio collected using each of the communication devices 72, 73, 74, and75. The server 20 transmits the video data and audio data, which arereceived from the communication devices 30, 71, 72, 73, 74, and 75, toappropriate destination devices, respectively. The destination devicerefers to each of the communication device 30 and the communicationdevices 71, 72, 73, 74, and 75 other than the sender device thattransmits video data and audio data. For example, the server 20transmits video data and audio data, which are received from thecommunication device 30, to each of the communication devices 71, 72,73, 74, and 75. The server 20 does not transmit the video data and theaudio data, which are received from the communication device 30, to thecommunication device 30, which is the sender device of the video dataand the audio data. In other words, in the remote conference system 10,the video data and audio data transmitted from the communication device30 are delivered to each of the communication devices 71, 72, 73, 74,and 75 via the server 20 using a streaming method. The video data andaudio data transmitted from each of the communication devices 71, 72,73, 74, and 75 are delivered to the communication device 30 via theserver 20 using the streaming method.

Video data may be moving image data compressed using a predeterminedcompression method. The compression method may be, for example, theH.264 video compression method. The video data according to theillustrative embodiment includes an intra-frame coded image data and aninter-frame coded image data, similar to known video data. A frame rateof video data may be, for example, 30 fps. The intra-frame coded imagedata includes an I-frame (intra-coded frame). The inter-frame codedimage data includes a P-frame (predicted frame) and a B-frame(bi-directional predicted frame). The I-frame is an image coded withoutusing inter-frame prediction. The P-frame is an image coded usingforward prediction. The B-frame is an image coded using one of forwardprediction, backward prediction, and bi-directional prediction. TheI-frame, the P-frame, the B-frame are compressed images already inpractical use. Therefore, a detailed description for the I-frame, theP-frame, the B-frame will be omitted. The I-frame, P-frame, and B-frameare also referred to irrespectively or correctively as “compressedimage”.

Each of the communication devices 30, 71, 72, 73, 74, and 75 transmitsits own identification (“ID”) information as sender-device's IDinformation, and a conference ID that identifies a remote conference inwhich each of the communication devices 30, 71, 72, 73, 74, and 75participates, along with video data and audio data. The server 20transmits the sender-device's ID information and the conference ID toeach destination device along with the video data and audio datareceived from the sender device. The ID information and conference IDare included in, for example, each header portion of the video data andthe audio data. Each ID information is information for identifyingparticular communication devices from one another. For example, the IDinformation of the communication device 30 is information foridentifying the communication device 30. The ID information of each ofthe communication devices 71, 72, 73, 74, and 75 is information foridentifying each of the communication devices 71, 72, 73, 74, and 75from one another. In the illustrative embodiment, it is assumed that allthe communication devices 30, 71, 72, 73, 74, and 75 participate in thesame remote conference that is identified by a predetermined conferenceID.

The communication device 30 obtains (e.g., receives) video data, audiodata, the ID information, and the conference ID transmitted from each ofthe communication devices 71, 72, 73, 74, and 75. In the communicationdevice 30, captured images are reproduced from the respective video datareceived from the communication devices 71, 72, 73, 74, and 75,respectively. In other words, the communication device 30 generatescaptured images 381, 382, 383, 384, and 385 corresponding to therespective video data. The communication device 30 also reproduces acaptured image from video data that it obtains. In other words, thecommunication device 30 generates a captured image corresponding to theobtained video data. The reproduction of a captured image may beperformed, for example, by decoding a compressed image included in videodata.

The captured image 381 corresponds to video data transmitted from thecommunication device 71. The captured image 382 corresponds to videodata transmitted from the communication device 72. The captured image383 corresponds to video data transmitted from the communication device73. The captured image 384 corresponds to video data transmitted fromthe communication device 74. The captured image 385 corresponds to videodata transmitted from the communication device 75.

The generated captured images 381, 382, 383, 384, and 385 and thecaptured image of the communication device 30 are laid out in accordancewith a layout setting. A predetermined program for remote conferenceincludes the layout setting. The predetermined program for remoteconference may be, for example, a program for receiving processing ofFIG. 10 (refer to step S77 in FIG. 10). The layout setting may be, forexample, information in which an entire area, including a partial areawhere one or more captured images including the captured images 381,382, 383, 384, and 385 are laid out, is patterned. In the communicationdevice 30, the layout of the captured images 381, 382, 383, 384, and 385and the captured image of the communication device 30 may be determinedarbitrarily in advance. In the communication device 30, one of theoptions “display” and “hide” may be set to each captured image. Forexample, when the option of “hide” is set to the captured image of thecommunication device 30, the captured image of the communication device30 is hidden. In the communication device 30, a remote conferencescreen, which includes contents thereon in accordance with the layoutsetting and the setting of one of the options of “display” and “hide”,is displayed (refer to FIG. 2).

In the illustrative embodiment, according to the layout setting, forexample, the option of “hide” is set to the captured image of thecommunication device 30 and the option of “display” is set to each ofthe captured images 381, 382, 383, 384, and 385. The captured images381, 382, 383, 384, and 385 are aligned in the horizontal direction(refer to FIG. 2). The entire area in which the captured images 381,382, 383, 384, and 385 are arranged in accordance with the layoutsetting is referred to as “video layout region A”. The captured image381 is arranged in an area A1 of the video layout region A. The capturedimage 382 is arranged in an area A2 of the video layout region A. Thecaptured image 383 is arranged in an area A3 of the video layout regionA. The captured image 384 is arranged in an area A4 of the video layoutregion A. The captured image 385 is arranged in an area A5 of the videolayout region A. The video layout region A includes one column by fiverows. The area A1 is an area in the “first column and first row” of thevideo layout region A. The area A2 is an area in the “first column andsecond row” of the video layout region A. The area A3 is an area in the“first column and third row” of the video layout region A. The area A4is an area in the “first column and fourth row” of the video layoutregion A. The area A5 is an area in the “first column and fifth row” ofthe video layout region A. In other embodiments, for example, the videolayout region A may include a plurality of columns.

The communication device 30 stores a status table therein (refer to FIG.3). Status information on the captured image 381 is stored in the statustable in association with the ID information of the communication device71. Status information on the captured image 382 is stored in the statustable in association with the ID information of the communication device72. Status information on the captured image 383 is stored in the statustable in association with the ID information of the communication device73. Status information on the captured image 384 is stored in the statustable in association with the ID information of the communication device74. Status information on the captured image 385 is stored in the statustable in association with the ID information of the communication device75. Each status information includes information relating to a displaycondition of a corresponding one of the captured images 381, 382, 383,384, and 385. Each status information includes information representingthe position of a corresponding one of the captured images 381, 382,383, 384, and 385 (hereinafter, referred to as “position information”).In an example depicted in FIG. 3, values “0”, “1” and “2” are definedfor the position information. The details of the values “0”, “1” and “2”will be described later. Each status information further includes atotal video play time of a corresponding one of the captured images 381,382, 383, 384, and 385. The total video play time indicates a totaldisplay duration of each of the captured images 381, 382, 383, 384, and385 in a video display region B during a remote conference. In thecommunication device 30, the duration for which each of the capturedimages 381, 382, 383, 384, and 385 is displayed is measured andaccumulated to obtain the total display duration.

In FIG. 3, ID information “71” indicates the ID information of thecommunication device 71. ID information “72” indicates the IDinformation of the communication device 72. ID information “73”indicates the ID information of the communication device 73. IDinformation “74” indicates the ID information of the communicationdevice 74. ID information “75” indicates the ID information of thecommunication device 75. The details relating to the display of thecaptured images 381, 382, 383, 384, and 385 that are laid out inaccordance with the layout setting will be further described later.

In each of the communication devices 71, 72, 73, 74, and 75, capturedimages are also reproduced from respective video data received from thecommunication devices 30, 71, 72, 73, 74, and 75 other than itself,respectively. In other words, each of the communication devices 71, 72,73, 74, and 75 generates captured images corresponding to the respectivevideo data. In each of the communication devices 71, 72, 73, 74, and 75,a remote conference screen, which includes contents thereon inaccordance with the layout setting and the setting of one of the optionsof “display” and “hide”, is displayed. The communication devices 71, 72,73, 74, and 75 are also configured to reproduce a captured image fromvideo data obtained by itself and display the generated captured imagethereon. Further, in each of the communication devices 71, 72, 73, 74,and 75, audio is reproduced from audio data received from each of thecommunication devices 30, 71, 72, 73, 74, and 75 other than itself andthe generated audio corresponding to each audio data is outputted.

In the illustrative embodiment, similar to the setting for thecommunication device 30, in the setting for the communication devices71, 72, 73, 74, and 75, the option of “hide” is set to own capturedimage generated by itself and the option of “display” is set to thecaptured images of the other communication devices. When a descriptionis made from the viewpoint of a device that obtains predetermined data,the “sender device” or “sender” of the predetermined data in theillustrative embodiment is also referred to as “supplier device” or“supplier” of the predetermined data.

<Server>

As depicted in FIG. 1, the server 20 includes a central processing unit(“CPU”) 22, a storage device 24, a random-access memory (“RAM”) 26, atimer 27, and a communication unit 28. The CPU 22, the storage device24, the RAM 26, the timer 27, and the communication unit 28 areconnected to a bus 29.

The CPU 22 is configured to execute calculation processing. The storagedevice 24 may be implemented by a computer-readable storage medium,e.g., a hard disk. In other embodiments, for example, the storage device24 may be implemented by a flash memory and/or a read-only memory(“ROM”). The storage device 24 stores therein various programs, forexample, an operating system (“OS”) and various applications. Theapplications stored in the storage device 24 include programs forexecuting various processing (refer to FIGS. 8, 9, 10, 15 and 16). Theprograms for executing the above processing are preinstalled in thestorage device 24.

The pre-installation of the execution programs is implemented by which areading unit (not depicted) of the server 20 reads the programs from acomputer-readable storage medium, e.g., a semiconductor memory. In otherembodiments, for example, when the server 20 includes an optical drive(not depicted), such a pre-installation may be implemented by which theoptical drive reads the programs from an optical medium. In still otherembodiments, for example, the pre-installation may be implemented bywhich the server 20 receives, via the communication unit 28 of theserver 20, the programs stored in a computer-readable storage medium,e.g., a hard disk, of another server, as transmission signals. The otherserver may be different from the server 20 connected to the network 90.The computer-readable storage medium may include a non-transitorystorage medium but not include a transitory storage medium. Thenon-transitory storage medium may include any storage medium that iscapable of storing information regardless of storage duration ofinformation.

The RAM 26 is a storage area to be used when the CPU 22 executes variousprograms. The RAM 26 stores, in a predetermined storage area,predetermined data and information used in various processing duringexecution of the various processing.

The CPU 22 is configured to control the server 20, for example, byexecuting the OS and the programs for executing the processing depictedin FIGS. 8, 9, 10, 15 and 16 stored in the storage device 24, wherebyvarious processing are executed and various functions are implemented inthe server 20.

The timer 27 has, for example, a calendar function and a clock function.The timer 27 is configured to measure elapsed time. In otherembodiments, for example, the timer 27 may be implemented by a clockfunction of the OS. The communication unit 28 is configured to connectthe server 20 to the network 90 and perform data communication via thenetwork 90. The server 20 is configured to receive video data, audiodata, and a particular conference ID from each communication device thatparticipates in a remote conference identified by the particularconference ID. The server 20 is further configured to transmit thereceived video data and audio data to each destination device. Forexample, the communication unit 28 receives video data and audio datatransmitted from each of the communication devices 30, 71, 72, 73, 74,and 75 and transmits the received video data and audio data to eachdestination device. In other words, the video data and audio datatransmitted from each of the communication devices 30, 71, 72, 73, 74,and 75 are transferred to each destination device via the communicationunit 28. The server 20 is hardwired to the network 90 via thecommunication unit 28. In other embodiments, for example, the server 20may be wirelessly connected to the network 90 via the communication unit28.

The server 20 is different from a known server in a point that thestorage device 24 of the server 20 stores the programs for executing theprocessing depicted in FIGS. 8, 9, 10, 15 and 16. The server 20 may bean information processing device having a communication function that isthe same as the known server in terms of hardware configuration. Inother embodiments, for example, the server 20 may have a configurationthat is the same as the known server.

<Communication Device>

As depicted in FIG. 1, the communication device 30 includes a CPU 32, astorage device 34, a RAM 36, a display 38, an operation unit 40, acamera 46, an audio unit 48, a timer 52, and a communication unit 54.The CPU 32, the storage device 34, the RAM 36, the display 38, theoperation unit 40, the camera 46, the audio unit 48, the timer 52, andthe communication unit 54 are connected to a bus 56.

The CPU 32 executes calculation processing. The storage device 34 may beimplemented by a computer-readable storage medium, e.g., a flash memory.In other embodiments, for example, the storage device 34 may beimplemented by a hard disk and/or a ROM. The storage device 34 storesvarious therein various programs, for example, an OS and variousapplications. The applications stored in the storage device 34 includeprograms for executing various processing (refer to FIGS. 4 and 7, 8, 9,and 10). The programs for executing the above processing may bepreinstalled in the storage device 34. In other embodiments, forexample, the programs may be transmitted to the communication device 30from the server 20 via the network 90 as transmission signals when thecommunication device 30 accesses the server 20 for participating in aremote conference. In this case, the programs are installed in thestorage device 34 or the RAM 36 of the communication device 30 uponparticipation of the remote conference.

The pre-installation of the execution programs is implemented by which areading unit (not depicted) of the communication device 30 reads theprograms from a computer-readable storage medium, e.g., a semiconductormemory. In other embodiments, for example, when the communication device30 includes an optical drive (not depicted), such a pre-installation maybe implemented by which the optical drive reads the programs from anoptical medium. In still other embodiments, for example, thepre-installation may be implemented by which the communication device 30receives, via the communication unit 54 of the communication device 30,the programs stored in a computer-readable storage medium, e.g., a harddisk, of another server, as transmission signals. The other server maybe different from the server 20 connected to the network 90. Thecomputer-readable storage medium may include a non-transitory storagemedium but not include a transitory storage medium. The non-transitorystorage medium may include any storage medium that is capable of storinginformation regardless of storage duration of information.

The RAM 36 is a storage area to be used when the CPU 32 executes variousprograms. The RAM 36 stores, in a predetermined storage area,predetermined data and information used in various processing duringexecution of the various processing. The RAM 36 also stores the statustable (refer to FIG. 3) therein.

The CPU 32 is configured to control the communication device 30, forexample, by executing the OS and the programs for executing theprocessing depicted in FIGS. 4, 5, 6, and 7 stored in the storage device34, whereby various processing are executed and various functions areimplemented in the communication device 30.

The display 38 is configured to display various information thereon. Forexample, the display 38 displays a remote conference screen includingthe captured images 381, 382, 383, 384, and 385 (refer to FIG. 2). Theoperation unit 40 is configured to receive an input, e.g., variousinstructions with respect to the communication device 30. The operationunit 40 includes, for example, a physical button 42 and a touch pad 44.The touch pad 44 may be, for example, a capacitive pointing device thatis configured to output a signal indicating a position of a coordinatecorresponding to a position where a finger of a user touches. In otherembodiments, for example, the touch pad 44 may be a resistive pointingdevice or an ultrasonic pointing device. The display 38 and the touchpad 44 (enclosed with a dashed line in FIG. 1) constitute a touch panel.

A user of the communication device 30 performs operations, e.g.,scrolling, swiping, flicking, tapping, dragging, pinch-in and/orpinch-out, on the touch pad 44. For example, the user of thecommunication device 30 moves a finger touching the touch pad 44 in apredetermined direction, to perform such operations. In response to theuser's operations, predetermined signals are outputted from the touchpad 44. Operation information (e.g., types of operations and movementdistance) corresponding to each operation is generated based on thepredetermined signals. Processing to generate the operation informationis employed in known smartphones or known tablet terminals asApplication Programming Interface (“API”) and also employed in thecommunication device 30. In other embodiments, for example, theoperation unit 40 may further include a keyboard and a mouse. When thecommunication device 30 is a personal computer, the operation unit 40includes a keyboard and a mouse.

The display of the captured images 381, 382, 383, 384, and 385 on thedisplay 38 will be described referring to FIG. 2. The display 38includes a display area. The video layout region A is an imaginary areathat is defined beyond the display area of the display 38. The videodisplay region B is a display area for remote conference that is definedwithin the display area of the display 38. In a case where an entireportion of the display area of the display 38 is defined as the displayarea for remote conference, the video display region B coincides withthe display area of the display 38. In the illustrative embodiment, itis assumed that the entire portion of the display area of the display 38is defined as the display area for remote conference. In FIG. 2, a“halftone area” of the display 38 indicates the display area of thedisplay 38. The captured images 381, 382, 383, 384, and 385 are laid outin the video layout region A. A width WA of the video layout region A inthe horizontal direction is greater than a width WB of the video displayregion B. A portion of the video layout region A is displayed in thevideo display region B. In other words, one or more, but not all, of thecaptured images 381, 382, 383, 384, and 385 are displayed in the videodisplay region B at one time. In an example depicted in FIG. 2,particular two of the captured images 381, 382, 383, 384, and 385 aredisplayed in the video display region B at one time.

For example, in order to change the display contents on the display areaof the display 38 from the captured images 381 and 382 (refer to anupper drawing in FIG. 2) to the captured images 384 and 385 (refer to alower drawing in FIG. 2), a user of the communication device 30 performsa swiping operation to move a finger touching the touch pad 44 from theright to the left. In response to this, the video layout region A movesrelative to the video display region B in the horizontal direction andthus the captured images 384 and 385 are displayed. In the communicationdevice 30, with reference to the horizontal direction, particular areasof the video layout region A displayed in the video display region B areidentified.

The position information to be stored in the status table (refer to FIG.3) will be described. The value “0” of the position informationindicates that a particular area of the video layout region A in which acaptured image corresponding to target ID information is arranged ispositioned within the video display region B. In other words, the value“0” of the position information indicates that the captured imagecorresponding to the target ID information is being displayed. Both thevalues “1” and “2” of the position information indicate that aparticular area of the video layout region A, in which a captured imagecorresponding to target ID information is arranged, is not positionedwithin the video display region B. In other words, the values “1” and“2” of the position information indicate that the captured imagecorresponding to the target ID information is being hidden. The value“1” of the position information indicates that a particular area of thevideo layout region A, in which a hidden captured image is positioned,within an area corresponding to a reference distance WC with referenceto a reference edge in a direction opposite to a moving direction. Thatis, when the position information indicates the value “1”, theparticular area in which the hidden captured image is arranged ispositioned within the area corresponding the reference distance WC withreference of the reference edge of the video display region B. The value“2” of the position information indicates that a particular area of thevideo layout region A in which a hidden captured image is arranged ispositioned out of the area corresponding to the reference distance WCwith reference to the reference edge in the direction opposite to themoving direction. That is, when the position information indicates thevalue “2”, the particular area in which the hidden captured image isarranged is positioned out of the area corresponding to the referencedistance WC with respect to the reference edge of the video displayregion B.

The moving direction of the captured images in response to a swipingoperation for displaying one or more hidden captured images is thehorizontal direction. In an upper drawing of FIG. 2, the hidden capturedimages may be, for example, the captured images 383, 384, and 385. In alower drawing of FIG. 2, the hidden captured images may be, for example,the captured images 381, 382, and 383. The reference edge may be atrailing edge of both edges of the video display region B in thehorizontal movement. For example, in the example depicted in the upperdrawing of FIG. 2, the right edge of the video display region B in thehorizontal direction may be the reference edge. In the example depictedin the lower drawing of FIG. 2, the left edge of the video displayregion B in the horizontal direction may be the reference edge. Thereference distance WC is a distance appropriate for a distance WD. Thedistance WD is a distance between edges on the same side (e.g., theright edges or the left edges) of adjacent two of the captured images inthe horizontal direction. In the illustrative embodiment, particular twoof a plurality of captured images are displayed in the video displayregion B at the same time (refer to FIG. 2). Therefore, the referencedistance WC is twice as long as the distance WD. In other embodiments,for example, the reference distance WC may be the same as the distanceWD or three times or more as long as the distance WD.

For example, in the example depicted in the upper drawing of FIG. 2, theareas A1 and A2 that are positioned within the video display region Bare identified, and the areas A3, A4, and A5 that are positioned out ofthe video display region B are identified. Further, the areas A3 and A4that are positioned within the area corresponding to the referencedistance WC with reference to the reference edge are identified amongthe areas A3, A4, and A5, and the area A5 that is positioned out of thearea corresponding to the reference distance WC with reference to thereference edge is identified. In the communication device 30, based onsuch identification of the particular areas, appropriate values arestored in the status table. In the illustrative embodiment, for example,the value “0” is stored as the position information in the status tablewith being associated with each of the sender's ID information “71” and“72” corresponding to the captured images 381 and 382 arranged in theareas A1 and A2 (refer to FIG. 3). The value “1” is stored as theposition information in the status table with being associated with eachof the sender's ID information “73” and “74” corresponding to thecaptured images 383 and 384 arranged in the areas A3 and A4 (refer toFIG. 3). The value“2” is stored as the position information in thestatus table with being associated with the sender's the ID information“75” corresponding to the captured image 385 arranged in the area A5(refer to FIG. 3).

In the example depicted in the lower drawing of FIG. 2, the areas A4 andA5 that are positioned within the video display region B are identified,and the areas A1, A2, and A3 that are positioned out of the videodisplay region B are identified. Further, the areas A2 and A3 that arepositioned within the area corresponding to the reference distance WCwith reference to the reference edge are identified among the areas A1,A2, and A3, and the area A1 that is positioned out of the areacorresponding to the reference distance WC with reference to thereference edge is identified. In this case, the value “0” is stored asthe position information in the status table with being associated witheach of the ID information “74” and “75”. The value “1” is stored as theposition information in the status table with being associated with eachof the ID information “72” and “73”. The value “2” is stored as theposition information in the status table with being associated with theID information “71”

The camera 46 is configured to capture an external image that presentsin a predetermined direction with respect to the communication device 30(e.g., in front of the communication device 30). For example, when thereis a user of the communication device 30 in front of the communicationdevice 30, the camera 46 captures an external image including the user.The audio unit 48 includes a speaker 49 and a microphone 50. The speaker49 is configured to output audio. The microphone 50 is configured tocollect external audio. For example, the microphone 50 collects voiceoutputted by the user of the communication device 30. The audio unit 48is configured to output audio corresponding to audio data from thespeaker 49. The audio unit 48 is configured to generate waveform datathrough analog-to-digital conversion of audio collected by themicrophone 50 using a predetermined sampling frequency (e.g., 11.025 kHzor 44.1 kHz). The communication device 30 is configured to startcapturing an external image using the camera 46 and collecting externalaudio using the microphone 50 upon start of a remote conference. In thecommunication device 30, video data corresponding to a captured imageand audio data corresponding to the generated waveform data aregenerated.

The timer 52 has, for example, a calendar function and a clock function.The timer 52 is configured to measure elapsed time. In otherembodiments, for example, the timer 52 may be implemented by a clockfunction of the OS. The communication unit 54 is configured to connectthe communication device 30 to the network 90 and perform datacommunication via the network 90. For example, the communication unit 54of the communication device 30 transmits video data and audio data,which are generated during a remote conference, to the server 20, alongwith the ID information of the communication device 30 and theconference ID identifying the remote conference in which thecommunication device 30 participates. Then, the server 20 furthertransmits the received video data, audio data, and ID information of thecommunication device 30 to each of the communication devices 71, 72, 73,74, and 75 that participate in the remote conference identified by thesame conference ID. The communication unit 54 of the communication unit30 receives video data, audio data, and the sender-device's IDinformation transmitted from each of the communication devices 71, 72,73, 74, and 75 via the server 20. The communication device 30 is wiredor wirelessly connected to the network 90 communication unit 54. Forexample, when the communication device 30 is a mobile device, thecommunication device 30 is wirelessly connected to the network 90 viathe communication unit 54. The communication unit 54 is a communicationmodule for performing wireless communication in compliance with a knowncommunication standard, for example, a Wi-Fi standard, a 4G standard, ora 3G standard.

The communication device 30 is different from a known mobile device in apoint that the storage device 34 of the communication device 30 storesthe programs for executing the processing depicted in FIGS. 4, 5, 6, and7. The communication device 30 may be a communication device that is thesame as a known mobile device in terms of hardware configuration.

<Processing Executed in Communication Device>

The various processing executed in the communication device 30 during aremote conference will be described. In the remote conference system 10,for example, the server 20 sends an electronic mail (“e-mail”) to e-mailaddresses corresponding to the communication devices 30, 71, 72, 73, 74,and 75 before a predetermined time and date of a particular remoteconference. Each e-mail includes a Uniform Resource Locator (“URL”) ofthe particular remote conference to be held among the communicationdevices 30, 71, 72, 73, 74, and 75. The URL is unique to each virtualconference room of a remote conference. In other words, the URL includesa conference ID of a remote conference. The conference ID may beincluded as, for example, a query parameter of a URL.

The user of the communication device 30 performs an appropriateoperation on the communication device 30 on or after the predeterminedtime and date of the particular remote conference. For example, in thecommunication device 30, the CPU 32 accesses the server 20 via thecommunication unit 54 based on the URL including the conference ID ofthe particular remote conference, and executes processing forestablishing a session for the particular remote conference with theserver 20. In other embodiments, for example, the CPU 32 may allow thecommunication unit 54 to transmit a login request including apredetermined user ID and a password to the server 20. In this case,when a login is successful, the CPU 32 may transmit the conference IDfrom the communication unit 54 to the server 20 to establish a sessionfor the particular remote conference with the server 20. Processingdescribed below are executed while the session for the particular remoteconference is established between the communication device 30 and theserver 20 (e.g., while the communication device 30 is connected with aparticular remote conference room for the particular remote conference).

The CPU 32 allows the timer 52 to start measuring time uponestablishment of a connection of the communication device 30 to theparticular remote conference room. The CPU 32 is configured to store, inthe RAM 36, elapsed time from the start of measurement. In otherembodiments, for example, the timer 52 may be configured to startmeasuring time upon establishment of connections of all thecommunication devices 30, 71, 72, 73, 74, and 75, which are scheduled toparticipate in the same remote conference identified by the sameconference ID, to the particular remote conference room. Uponestablishment of connections of all the communication devices 30, 71,72, 73, 74, and 75, which are scheduled to participate in the sameremote conference, to the particular remote conference room, the server20 notifies each of the communication devices 30, 71, 72, 73, 74, and 75that all the communication devices 30, 71, 72, 73, 74, and 75 haveconnected to the particular remote conference room.

<Audio Data Transmitting Processing>

Audio Data transmitting processing will be described referring to FIG.4. The audio data transmitting processing is repeatedly executed atregular intervals until the communication device 30 is disconnected fromthe particular conference room after establishing the connectionthereto. The audio data transmitting processing is repeatedly executed,for example, at predetermined encoding intervals (e.g., at intervals of20 msec.). The play duration of 20 msec. of audio data corresponds tothe encoding intervals of 20 msec. Subsequent to starting the audio datatransmitting processing, the CPU 32 obtains audio collected using themicrophone 50 (e.g., step S11). The audio obtained by the CPU 32 in stepS11 includes waveform data generated through the analog-to-digitalconversion of audio collected by the microphone 50 using thepredetermined sampling frequency (e.g., 11.025 kHz or 44.1 kHz). The CPU32 determines volume (dB) of the collected audio and judges whether thevalue representing the determined volume of the collected audio isgreater than a threshold value (e.g., step S13). For example, the volumemay be determined by determination of a level of the waveform of theaudio obtained in step S11. The audio collected in step S11 includes aplurality of sampling points. Therefore, for example, an average levelof a plurality of sampling points included in a predetermined timeperiod may be determined as volume of audio. In other embodiments, forexample, a maximum level of a plurality of sampling points included inthe predetermined time period may be determined as volume of audio. Thethreshold value, which is the reference to be used in step S13, ispredetermined in consideration given to volume of user's voice such thatthe CPU 32 determines that the user of the communication device 30outputs voice in a remote conference. The threshold value is stored inthe storage device 34 with being associated with the program forexecuting audio data transmitting processing. In other embodiments, forexample, the threshold value may be changed in response to degree ofbackground noises.

When the value representing the determined volume of audio is greaterthan the threshold value (e.g., YES in step

13), the CPU 32 generates compressed audio data by encoding the obtainedaudio using a predetermined compression method, e.g., MPEG-4 AAC orG.711 (e.g., step S15). The audio data may be a packet that includes,for example, encoded data corresponding waveform data having a playduration of 20 msec. The CPU 32 assigns the ID information of thecommunication device 30 to the generated audio data (e.g., step S17).Subsequently, the CPU 32 controls transmission of the audio dataassigned with the ID information (e.g., step S19). In step S19, the CPU32 outputs, to the communication unit 54, an instruction to transmit theaudio data. Thus, the audio data is transmitted from the communicationunit 54 to the server 20. When the value representing the determinedvolume of audio is smaller than or equal to the threshold value (e.g.,NO in step S13), or subsequent to step S19, the CPU 32 ends the audiodata transmitting processing.

<Video Data Transmitting Processing>

Video data transmitting processing will be described referring to FIG.5. The video data transmitting processing is repeatedly executed atregular intervals until the communication device 30 is disconnected fromthe particular conference room after establishing the connectionthereto. The video data transmitting processing is repeatedly executedat intervals corresponding to the frame rate of video data. For example,when the frame rate is 30 fps, the video data transmitting processing isexecuted at intervals of 1/30 sec. Subsequent to starting the video datatransmitting processing, the CPU 32 obtains a captured image capturedusing the camera 46 (e.g., step S21). Subsequently, the CPU 32 generatesa compressed image by encoding the captured image using a predeterminedcompression method, e.g., the H.264 video compression method (e.g., stepS23).

The CPU 32 assigns image type information and the ID information of thecommunication device 30 to video data including the generated compressedimage (e.g., step S25). The image type information indicates a type ofthe generated compressed image. For example, when the generatedcompressed image is an I-frame, the video data is assigned with theimage type information indicating I-frame. When the generated compressedimage is a P-frame, the video data is assigned with the image typeinformation indicating P-frame. When the generated compressed image is aB-frame, the video data is assigned with the image type informationindicating B-frame.

The CPU 32 controls transmission of the video data including thecompressed image. The vide data is assigned with the ID information ofthe communication device 30 and the image type information (e.g., stepS27). In step S27, the CPU 32 outputs, to the communication unit 54, aninstruction to transmit the video data. Thus, the video data istransmitted from the communication unit 54 to the server 20. Subsequentto step S27, the CPU 32 ends the video data transmitting processing.

<Receiving Processing>

Receiving processing will be described referring to

FIG. 6. The receiving processing is repeatedly executed at regularintervals until the communication device 30 is disconnected from theparticular conference room after establishing the connection thereto.Subsequent to starting the receiving processing, the CPU 32 judgeswhether the predetermined data transmitted from the server 20 has beenobtained (e.g., received via the communication unit 54) (e.g., stepS31). When the CPU 32 judges that the predetermined data has not beenobtained yet (e.g., NO in step S31), the CPU 32 repeatedly executes thisjudgment step.

When the CPU 32 judges that the predetermined data has received (e.g.,YES in step S31), the CPU 32 judges whether the received data is audiodata (e.g., step S33). When the CPU 32 judges that the received data isaudio data (e.g., YES in step 33), the CPU 32 controls output of audiocorresponding to the obtained audio data (e.g., step S35). In step S35,the CPU 32 reproduces audio from the obtained audio data (e.g., decodesthe received audio data) and outputs, to the audio unit 48, aninstruction to output the reproduced audio. In response to the outputinstruction, the audio unit 48 outputs audio corresponding to the audiodata through the speaker 49. When audio data transmitted from thecommunication device 71 has been received while the display 38 is in astate depicted in the upper drawing of FIG. 2, audio including voice ofa user of the communication device 71 is outputted through the speaker49. When audio data transmitted from the communication device 74 hasbeen received while the display 38 is in a state depicted in the upperdrawing of FIG. 2, audio including voice of a user of the communicationdevice 74 is outputted through the speaker 49. In other words, in thecommunication device 30, audio corresponding to each audio data receivedfrom each of the communication devices 71, 72, 73, 74, and 75 isoutputted through the speaker 49 regardless of whether the capturedimages 381, 382, 383, 384, and 385 of the communication devices 71, 72,73, 74, and 75 are displayed or hidden. Subsequent to step S35, theroutine returns to step S31 and the CPU 32 executes step S31 andsubsequent steps again.

When the received data is not audio data (e.g., NO in step S33), the CPU32 judges whether the received data is video data (e.g., step S37). Whenthe received data is not video data (e.g., NO in step S37), the CPU 32executes processing appropriate for the received data as necessary.Subsequently, the routine returns to step S31 and the CPU 32 executesstep S31 and subsequent steps again.

When the received data is audio data (e.g., YES in step S37), the CPU 32obtains the ID information of the communication device that is a senderof the video data (e.g., step S39). For example, when the received datais video data transmitted from the communication device 71, the CPU 32obtains the ID information of the communication device 71 assigned tothe video data. When the received data is video data transmitted fromthe communication device 75, the CPU 32 obtains the ID information ofthe communication device 75.

The CPU 32 determines a position of the display area for the receivedvideo data in the video layout region A (e.g., step S41). In oneexample, it is assumed that audio data transmitted from thecommunication device 71 has been received while the display 38 is in thestate depicted in the upper drawing of FIG. 2. In step S41, as a firststep, the CPU 32 identifies the areas A1 and A2 that positioned withinthe video display region B. As a second step, the CPU 32 identifies thearea A1 in which the captured image 381 corresponding to video datatransmitted from the communication device 71 is arranged in the videolayout region A, based on the layout setting. In the communicationdevice 30, each of the areas A1, A2, A3, A4, and A5 included in thevideo layout region A is stored with being associated with correspondingID information. Based on the ID information obtained in step S39, theCPU 32 identifies a particular area of the video layout region Aarranged in accordance with the layout setting. Then, the CPU 32determines whether the particular area is identified in both the firstand second steps. In this case, the area A1 is identified in both thefirst and second steps. Therefore, the CPU 32 determines that the areaA1 is positioned within the video display region B.

In another example, it is assumed that video data transmitted from thecommunication device 75 has been received while the display 38 is in thestate depicted in the upper drawing of FIG. 2. In this case, in stepS41, as a first step, the CPU 32 identifies the areas A1 and A2 thatpositioned within the video display region B. As a second step, the CPU32 identifies the area A5 in which the captured image 385 correspondingto video data transmitted from the communication device 75 is arrangedin the video layout region A, based on the layout setting. In this case,the area A5 is not identified in both the first and second steps.Therefore, the CPU 32 determines that the area A5 is positioned out ofthe video display region B.

Subsequently, the CPU 32 controls the display condition according to thelayout setting (e.g., step S43). During this control, the CPU 32executes processing based on the determination result made in step S43.For example, when the received data is video data transmitted from thecommunication device 71, the CPU 32 controls the display condition ofthe captured image 381 based on the determination result that thereceived video data is positioned within the video display region B. TheCPU 32 reproduces video based on the video data transmitted from thecommunication device 71 and obtains the captured image 381.Subsequently, the CPU 32 outputs, to the display 38, an instruction tooutput the captured image 381. In response to the output instruction,the display 38 displays the captured image 381 newly obtained, in thearea A1. Subsequent to step S43, the routine returns to step S31 and theCPU 32 executes step S31 and subsequent steps.

When the determination result made in step S41 indicates that thereceived video data is positioned out of the video display region B, theroutine skip step S43. For example, when the received data is video datatransmitted from communication device 75, the CPU 32 does not executethe control of the display condition of the captured image 385corresponding to video data transmitted from the communication device75. Subsequent to step S41, the routine skips step S43 and returns tostep S31.

<Layout Information Transmitting Processing>

Layout information transmitting processing will be described referringto FIG. 7. The layout information transmitting processing is repeatedlyexecuted at regular intervals until the communication device 30 isdisconnected from the particular conference room after establishing theconnection thereto and displaying a remote conference screen (refer toFIG. 2) including one or more of the captured images 381, 382, 383, 384,and 385 in response to step S43 of FIG. 6. The layout informationtransmitting processing is executed, at every predetermined interval,e.g., 100 msec. Subsequent to starting the layout informationtransmitting processing, the CPU 32 specifies a positional relationshipbetween the video layout region A and the video display region B (e.g.,step S51). For example, the CPU 32 identifies particular two areas ofthe video layout region A positioned within the video display region Band the other areas of the video layout region A positioned out of thevideo display region B. In the case depicted in the upper drawing ofFIG. 2, the CPU 32 identifies the areas A1 and A2 of the video layoutregion A that are positioned within the video display region B and theareas A3, A4, and A5 of the video layout region A that are positionedout of the video display region B.

Subsequently, the CPU 32 generates layout information (e.g., step S53).At the generation of the layout information, the CPU 32 updates eachinformation stored in the status table (refer to FIG. 3). The CPU 32identifies the ID information associated with each of the particularareas, among the areas that are arranged in the video layout region A inaccordance with the layout setting. The particular areas have beenidentified in step S51 as such that the particular areas are positionedwithin the video display region B. Then, the CPU 32 designates the value“0” as the position information in association with the identified IDinformation. The CPU 32 further identifies one or more particular areaspositioned within the area corresponding to the reference distance WCand one or more other particular areas positioned out of the areacorresponding to the reference distance WC, among the areas that arearranged in the video layout region A in accordance with the layoutsetting and have been identified in step S51 as such that the areas arepositioned out of the video display region B. Then, the CPU 32identifies the ID information associated with each of the particularareas positioned within the area corresponding to the reference distanceWC and the ID information associated with each of the particular areaspositioned out of the area corresponding to the reference distance WC.Then, the CPU 32 designates the value “1” as the position information inassociation with the ID information identifying the area positionedwithin the area corresponding to the reference distance WC. The CPU 32also designates the value “2” as the position information in associationwith the ID information identifying the area positioned out of the areacorresponding to the reference distance WC. At the generation of thelayout information in step S53, the CPU 32 updates the total video playtime associated with the ID information associated with the area that isidentified in step S51 as the area of the video layout region Apositioned within the video display region B.

For example, as described above, it is assumed that the areas A1 and A2are identified as the areas of the video layout region A positionedwithin the video display region B (refer to the upper drawing of FIG.2). In this case, the CPU 32 updates the position information in thestatus table as described in FIG. 3. Further, the CPU 32 adds apredetermined amount of time (e.g., 100 msec.), which is thepredetermined interval, to each total video play time associated with acorresponding one of the ID information “71” and “72”. The predeterminedinterval may be the time elapsed between a timing of the last executionof step S53 and a timing of this-time execution of step S53.

In step S53, the CPU 32 generates layout information, which correspondsto each information and the associations stored in the status table,based on the status table updated as described above. The layoutinformation includes the status information and the ID informationassociated with each other. Subsequently, the CPU 32 controlstransmission of the layout information and the own ID information (e.g.,step S55). In S55, the CPU 32 outputs, to the communication unit 54, aninstruction to transmit the layout information and the ID information ofthe communication device 30 to the server 20. Thus, the update frequencysetting and the ID information of the communication device aretransmitted from the communication unit 54 to the server 20. The IDinformation of the communication device 30 that is a sender may beincluded in, for example, a header portion of the layout information.Subsequent to step S55, the CPU 32 ends the layout informationtransmitting processing.

<Processing Executed in Server>

Processing executed in the server 20 during the remote conference willbe described. Each processing described below is executed for one ormore of the communication devices 30, 71, 72, 73, 74, and 75, which areconnected to the same conference room. The CPU 22 starts measuring anelapsed time using the timer 27 at a timing at which one of thecommunication devices 30, 71, 72, 73, 74, and 75 connects to aparticular remote conference room. The CPU 22 stores the measured timeelapsed from the start of measurement in the RAM 26. In otherembodiments, for example, the time measurement using the timer 27 may bestarted at a timing at which all of the predetermined communicationdevices 30, 71, 72, 73, 74, and 75, connect to a particular remoteconference room for a remote conference identified by a conference ID.In description of processing executed in server 20, one or more or allof the communication devices 30, 71, 72, 73, 74, and 75 may refer to“communication device or communication devices” without referencenumerals.

<Transferring Processing>

Transferring processing will be described referring to FIGS. 8, 9, and10. The transferring processing is repeatedly executed until all of oneor more of the communication devices 30, 71, 72, 73, 74, and 75disconnect from the particular conference room after one of the one ormore communication devices 30, 71, 72, 73, 74, and 75 connects to theparticular conference room. Subsequent to starting the transferringprocessing, the CPU 22 judges whether the predetermined data transmittedfrom one of the communication devices 30, 71, 72, 73, 74, and 75 hasbeen obtained (e.g., received via the communication unit 28) (e.g., stepS61). When the CPU 22 judges that the predetermined data has not beenreceived (e.g., NO in step S61), the CPU 22 executes this determinationstep repeatedly.

When the CPU 22 judges that the predetermined data has been received(e.g., YES in step S61), the CPU 22 judges whether the received data islayout information (e.g., step S63). When the CPU 22 judges that thereceived data is not layout information (e.g., NO in step S63), theroutine proceeds to step S75 of FIG. 9. When the CPU 22 judges that thereceived data is layout information (e.g., YES in step S63), the CPU 22stores the layout information (e.g., step S65). The layout informationis stored in the RAM 26 with being associated with the ID information ofthe communication device that is the sender of the layout information(refer to FIG. 11). The sender-device's ID information is received bythe communication unit 28 along with the layout information. The CPU 22obtains the sender-device's ID information along with the layoutinformation via the communication unit 28.

In FIG. 11, “ID information (target)” corresponds to the “ID information(target)” included in the layout information generated in step S53 ofFIG. 7 based on the status table depicted in FIG. 3. In the illustrativeembodiment, each communication device obtains a captured image of owndevice from video data obtained in itself (e.g., directly from camera46) without the captured image passing through the server 20. Therefore,the layout information transmitted from each of the communicationdevices, i.e., the layout information of the sender device, might notinclude the status information for own device. Although the layoutinformation transmitted from each communication device is stored in theserver 20 (refer to FIG. 11), there is no layout information for the “IDinformation (target)” corresponding to the sender-device's IDinformation. The layout information of the communication device 30 istransmitted from the communication device 30 in step S55 of FIG. 7. Thelayout information of each of the communication devices 71, 72, 73, 74,and 75 is transmitted from each of the communication devices 71, 72, 73,74, and 75 in a step corresponding to step S55 of FIG. 7.

Subsequently, the CPU 22 judges whether steps S69 to S73 have beenexecuted for all of each of the communication devices identified by the“ID information (target)” included in the layout information stored instep S65 after received (e.g., step S67). The processing of steps S69 toS73 are a series of processing in which update frequency information isdetermined for each communication device identified by the “IDinformation (target)” included in the layout information stored in stepS65 and the update frequency information determined for eachcommunication device is stored in the update frequency settingassociated with the ID information of the communication device that isthe sender of the layout information. When the CPU 22 judges that thereis no communication device on which steps S69 to S73 have not beenexecuted yet, i.e., steps S69 to S73 have already been executed for allthe communication devices (e.g., YES in step S67), the routine returnsto step S61 and the CPU 22 executes the processing of S61 and subsequentsteps. When the CPU 22 judges that there is one or more communicationdevices on which steps S69 to S73 have not been executed yet, i.e.,steps S69 to S73 have not been executed for all the communicationdevices (e.g., NO in step S67), the CPU 22 determines one of the one ormore communication devices as a target to be processed (e.g., step S69).For example, it is assumed that the layout information stored in stepS65 is the layout information transmitted from the communication device30 and steps S69 to S73 have been executed for the communication devices71, 72, and 73 but not executed yet for the communication devices 74 and75. In this case, one of the communication devices 74 and 75, e.g., thecommunication device 74, is determined as a target to be processed.

Subsequently, the CPU 22 executes the update frequency determiningprocessing (e.g., step S71). The update frequency determining processingis executed for the communication device selected in step S69. Theupdate frequency determining processing will be described later. The CPU22 stores the determined update frequency information for thecommunication device selected in step S69 to the update frequencysetting that is associated with the ID information of the sender devicethat transmits the layout information (e.g., step S73). The updatefrequency information is determined in the update frequency determiningprocessing that is executed for the communication device selected instep S69. The sender-device's ID information is received along with thelayout information. The update frequency setting is stored in the RAM 26with being associated with the ID information of the sender device thattransmits the layout information (refer to FIG. 12). Subsequent to stepS73, the routine returns to step S67 and the CPU 22 executes step S67and subsequent steps.

The update frequency information indicates whether which one or more ofcompressed images of various types included in video data are determinedas a non-target for transmission. For example, the update frequencyinformation is defined as described in FIG. 13. An update ruleconcerning the update frequency information will be described referringto FIG. 5. In the illustrative embodiment, for example, the update ruledefines update frequency levels “5”, “4”, “3”, “2”, and “1” for theupdate frequency information. The update frequency level “5” is thehighest level for the update frequency information. Subsequent to theupdate frequency level “5”, the level becomes lower in the order of theupdate frequency levels “4”, “3”, and “2”. The update frequency level“1” is the lowest level for the update frequency information.

At the update frequency level “5”, for example, an I-frame, a P-frame,and a B-frame are all determined as a target for transmission. None ofthe I-frame, the P-frame, and the B-frame is determined as a non-targetfor transmission. Therefore, for video data corresponding to the updatefrequency level “5”, I-frames, P-frames, and B-frames are alltransmitted to each destination device, other than the sender device ofthe video data, via the server 20. Thus, this rule enables a capturedimage to be updated in the communication device 30 based on all of theI-frames, the P-frames, and the B-frames. A minimum I-frame transmissioninterval is defined as 0 msec. The minimum I-frame transmission intervalindicates the shortest interval at which the server 20 transmits anI-frame which is a target for transmission.

At the update frequency level “4”, for example, an I-frame and a P-frameare determined as a target for transmission and a B-frame is determinedas a non-target for transmission. Therefore, for video datacorresponding to the update frequency level “4”, I-frames and P-framesare transmitted to each destination device, other than the sender deviceof the video data, via the server 20, and B-frames are not transmittedto the communication device 30. Thus, this rule enables a captured imageto be updated in the communication device 30 based on the I-frames andthe P-frames. The minimum I-frame transmission interval is defined as 0msec.

At the update frequency level “3”, for example, an I-frame is determinedas a target for transmission and a P-frame and a B-frame are determinedas a non-target for transmission. The minimum I-frame transmissioninterval is defined as 300 msec. Therefore, for video data correspondingto the update frequency level “3”, I-frames are transmitted to eachdestination device, other than the sender device of the video data, viathe server 20 at intervals of 300 msec. or longer, and P-frames andB-frames are not transmitted to the communication device 30. Thus, thisrule enables a captured image to be updated in the communication device30 based on the I-frames transmitted from the server 20 at intervals of300 msec. or longer.

At the update frequency level “2”, for example, an I-frame is determinedas a target for transmission and a P-frame and a B-frame are determinedas a non-target for transmission. The minimum I-frame transmissioninterval is defined as 2000 msec. Therefore, for video datacorresponding to the update frequency level “2”, I-frames aretransmitted to each destination device, other than the sender device ofthe video data, via the server 20 at intervals of 2000 msec. or longer,and P-frames and B-frames are not transmitted to the communicationdevice 30. This rule enables a captured image to be updated in thecommunication device 30 based on I-frames transmitted from the server 20at intervals of 2000 msec. or longer.

At the update frequency level “1”, for example, an I-frame, a P-frame,and a B-frame are all determined as a non-target for transmission. Inthis case, transmission of video data corresponding to the updatefrequency level “1” to each destination device, other than the senderdevice of the video data, via the server 20 is stopped. The illustrativeembodiment will be described below using example cases according to theupdate frequency levels “5”, “4”, “3”, “2”, and “1” depicted in FIG. 13.

The processing of S73 will be described assuming that the sender of thelayout information stored in step S65 is the communication device 30,the communication device 74 is selected as a target to be processed instep S69, and the update frequency level “1” is determined as the updatefrequency information in the update frequency determining processing ofstep S71. The CPU 22 stores the update frequency level “1” for the IDinformation “74” identifying the communication device 74 to be processedin association with the ID information “30” identifying thecommunication device 30 that is the sender of the layout information(refer to FIG. 12).

In FIG. 12, the “ID information (target)” corresponds to the “IDinformation (target)” included in the layout information. As describedabove, the layout information does not include the status information ofthe sender device. Therefore, there is no update frequency informationstored for the “ID information (target)” corresponding to thesender-device's ID information with respect to the update frequencysetting stored in the server 20 (refer to FIG. 12). The values indicatedin the status information (e.g., the position information and the totalvideo play time) depicted in FIG. 11 and the values indicated in theupdate frequency information depicted in FIG. 12 are merely arbitraryexample values, and consideration might not be given to correspondencesamong the values in the status information and in the update frequencyinformation.

The CPU 22 judges whether the received data is video data (e.g., stepS75). When the received data is not video data (e.g., NO in step S75),the routine proceeds to step S97 of FIG. 10. When the received data isvideo data (e.g., YES in step S75), the CPU 22 obtains the IDinformation and the image type information which are assigned to thevideo data (e.g., step S77). The video data received from thecommunication device 30 is the video data transmitted from thecommunication device 30 in step S27 of FIG. 5. The video data of thecommunication devices 71, 72, 73, 74, and 75 received from thecommunication devices 71, 72, 73, 74, and 75 are the video datatransmitted from the communication devices 71, 72, 73, 74, and 75,respectively, in a step corresponding to step S27 of FIG. 5. The CPU 22stores the obtained ID information and compressed image information inthe RAM 26.

The CPU 22 judges whether the compressed image included in the videodata is I-frame (e.g., step S79). This judgment is made based on theimage type information obtained in step S77. When the CPU 22 judge thatthe compressed image is I-frame (e.g., YES in step S79), the CPU 22obtains bandwidth provided for the sender device of the video data(e.g., step S81). The bandwidth is calculated by dividing a total dataamount of all compressed image data received between after last timereception of I-frame and before this time reception of I-frame in onecommunication device by a time elapsed between after last time receptionof I-frame and before this time reception of I-frame.

In order to calculate the bandwidth, the CPU 22 obtains the total dataamount and the elapsed time. The total data amount is obtained, forexample, as described below. Once the CPU 22 obtains an I-frame from apredetermined communication device, every time the CPU 22 obtains acompressed image from the predetermined communication device, the CPU 22adds an amount of data of the received compressed image. The adding ofthe amount of data is continued until the CPU 22 receives anotherI-frame from the predetermined communication device. When the CPU 22newly receives an I-frame from the predetermined communication device,the CPU 22 adds the amount of data of the newly-received I-frame to anaccumulated value to obtain the total data amount. The CPU 22 resets theaccumulated value as the CPU 22 obtains the total data amount.Subsequent to resetting the accumulated value, the CPU 22 starts addinga data mount of a received compressed image. The elapsed time isobtained based on a difference between a timing at which the CPU 22receives an I-frame last time and a timing at which the CPU 22 receivesanother I-frame this time in accordance with the storing executed instep S95. The CPU 22 stores the bandwidth obtained in step S81 in theRAM 26 with being associated with the ID information obtained in stepS77 of FIG. 9.

When the CPU 22 judges that the compressed image is not I-frame (e.g.,NO in step S79), or subsequent to step S81, the CPU 22 judges whethersteps S85 to S95 have already been executed for all the communicationdevices that are connected to the particular conference room, other thanthe communication device identified by the ID information obtained instep S77 (e.g., step S83). Steps S85 to S95 are a series of processingfor transmitting video data selectively. The CPU 22 judges, for example,whether the received video data (e.g., YES in step S75) is video data tobe transmitted for each destination device on communication-devicebasis. When the CPU 22 judges that the received video data is video datato be transmitted for each destination device, the CPU 22 transmits thevideo data to one or more destination devices. Steps S85 to S95 areexecuted for all the communication devices that are connected to theparticular conference room, other than the communication deviceidentified by the ID information that has been obtained in step S77.When the CPU 22 judges that there is no communication device on whichsteps S85 to S95 have not been executed yet, i.e., steps S85 to S95 havealready been executed for all the communication devices (e.g., YES instep S83), the routine returns to step S61 of FIG. 8 and the CPU 22executes processing of step S61 and subsequent steps. When the CPU 22judges that there is one or more communication devices on which stepsS85 to S95 have not been executed yet, i.e., steps S85 to S95 have notbeen executed for all the communication devices (e.g., NO in step S83),the CPU 22 determines one of the one or more communication devices as atarget to be processed (e.g., step S85). For example, it is assumed thatthe received video data is video data transmitted from the communicationdevice 71, i.e., the CPU 22 has obtained the ID information of thecommunication device 71 in step S77. When steps S85 to S95 have not yetbeen executed for the communication devices 30 and 75 although steps S85to S95 have already been executed for the communication devices 72, 73,and 74, one of the communication devices 30 and 75, e.g., thecommunication device 30, is determined as a target to be processed.

Subsequently, the CPU 22 executes a transfer determining processing(e.g., step S89). A detail of the transfer determining processing willbe described later. Subsequent to the transfer determining processing,the CPU 22 judges whether the determination result stored in the RAM 26indicates that it is necessary to transfer the received video data(hereinafter, referred to as “transfer necessary”) (e.g., step S91).When the CPU 22 judges that the determination result indicates “transfernecessary” (e.g., YES in step S91), the CPU 22 controls transmission ofthe received video data (e.g., step S93). The communication device thatis a target to be processed is determined as the destination of thevideo data. Subsequently, the CPU 22 stores the ID information of thesender device that has transmitted the video data, the ID information ofthe destination device, a transmission time, and the image typeinformation, for the transmission in step S93 (e.g., step S95). Thesender-device's ID information, the destination-device's ID information,the transmission time, and the image type information are stored in atransmission table (not depicted) stored in the RAM 26, for example.

Details of steps S93 and S95 will be described below assuming that thereceived video data is video data transmitted from the communicationdevice 71, the ID information of the communication device 71 is obtainedin step S77, and the communication device 30 is determined as a targetto be processed in step S85. In step S93, the CPU 22 determines thecommunication device 30 as a destination of the video data and outputs,to the communication unit 28, an instruction to transmit the video datareceived from the communication device 71. Thus, the video data receivedfrom the communication device 71 is transmitted to the communicationdevice 30 from the communication unit 28. In step S95, the CPU 22 storesthe ID information of the communication device 71, the ID information ofthe communication device 30, a transmission time, and the image typeinformation in the transmission table stored in the RAM 26.

When the CPU 22 judges that the determination result indicates it isunnecessary to transfer the received video data (hereinafter, referredto as “transfer unnecessary”) (e.g., NO in step S91), or subsequent tostep S95, the routine returns to step S83 and the CPU 22 executes stepS83 and subsequent steps.

In step S97 of FIG. 10, the CPU 22 judges whether the received data isaudio data. When the CPU 22 judges that the received data is not audiodata (e.g., step NO in step S97), the CPU 22 executes processingappropriate for the received data. Subsequently, the routine returns tostep S61 of FIG. 8 and the CPU 22 executes step S61 and subsequentsteps.

When the CPU 22 judges that the received data is audio data (e.g., YESin step S97), the CPU 22 obtains the ID information assigned to theaudio data (e.g., step S99). The audio data transmitted from thecommunication device 30 is the audio data transmitted from thecommunication device 30 in step S19 of FIG. 4. The audio datatransmitted from each of the communication devices 71, 72, 73, 74, and75 is the audio data transmitted by each of the communication devices71, 72, 73, 74, and 75 in a step corresponding to step S19 of FIG. 4.

Subsequently, the CPU 22 updates the total audio play time associatedwith the received ID information (e.g., step S101). The total audio playtime indicates a total play duration of audio data transmitted from eachof the communication devices 30, 71, 72, 73, 74, and 75 during a remoteconference. Audio data is transmitted and received in unit of a blockhaving a play duration of a predetermined time via the server 20 in theremote conference system 10. In other words, audio data is transmittedto the server 20 from each communication device at a certainpredetermined transmission interval. For example, audio data istransmitted and received in unit of a block having a play duration of 20milliseconds (msec.). The play duration of 20 msec. of audio datacorresponds to the encoding intervals of 20 msec., which is the encodinginterval for repeating the audio data transmitting processing (refer toFIG. 4). In the server 20, the total audio play time is stored in atotal audio play time table with being associated with the IDinformation of a corresponding one of the communication devices (referto FIG. 14). The total audio play time table is stored in, for example,the RAM 26. Each total audio play time indicates a sum of the play timeof audio data transmitted from a corresponding one of the communicationdevices.

In step S101, the CPU 22 accesses the total audio play time table storedin the RAM 26 and adds the play time to the total audio play timeassociated with the received ID information. In an example, it isassumed that audio data has a play time of 20 msec. and the IDinformation of the communication device 71 has been obtained in stepS99. In this case, the CPU 22 adds 0.02 sec to the total audio play timeassociated with the ID information of the communication device 71 in thetotal audio play time table. In another example, it is assumed thataudio data has a play time of 20 msec. and the ID information of thecommunication device 74 has been obtained in step S99. In this case, theCPU 22 adds 0.02 sec to the total audio play time associated with the IDinformation of the communication device 74 in the total audio play timetable.

Subsequently, the CPU 22 controls transmission of the audio data (e.g.,step S103). All of the one or more communication devices identified bythe respective ID information, other than the ID information of thecommunication device assigned to the audio data, are determined as adestination of the audio data. For example, it is assumed that thereceived data is audio data transmitted from the communication device30. In this case, the CPU 22 determines the communication devices 71,72, 73, 74, and 75 identified by the respective ID information that aredifferent from the ID information of the communication device 30assigned to the audio data, as a destination of the video data.Subsequently, the CPU 22 outputs, to the communication unit 28, aninstruction to transmit the audio data received from the communicationdevice 30. Thus, the audio data received from the communication device30 is transmitted to each of the communication devices 71, 72, 73, 74,and 75 from the communication unit 28. Subsequent to step S103, theroutine returns to step S61 of FIG. 8 and the CPU 22 executes step S61and subsequent steps.

<Update Frequency Determining Processing>

Update frequency determining processing, which is executed at step S71in the transferring processing depicted in FIGS. 8, 9, and 10, will bedescribed referring to FIG. 15 (refer to FIG. 8). The update frequencydetermining processing is executed for the communication device that isselected in step S69 of FIG. 8. In the illustrative embodiment, theupdate frequency determining processing is executed five times for fivecommunication devices, other than the communication device that is thesender of the layout information stored in step S65 of FIG. 8, everytime the layout information is stored in step S65 (e.g., YES in step S67of FIG. 8). The CPU 22 executes the update frequency determiningprocessing for one target among the communication devices 30, 71, 72,73, 74, and 75 to determine update frequency information for the targetcommunication device.

The CPU 22 judges whether the position information indicates the value“0”, which indicates that a particular captured image is being displayed(e.g., step S111). The position information is included in the statusinformation associated with the ID information of the communicationdevice that is selected in step S69 and is targeted for the updatefrequency determining processing. In step S111, the CPU 22 accesses thelayout information stored in step S65 of FIG. 8 and executes thedetermination based on the layout information. For example, it isassumed that the layout information transmitted from the communicationdevice 30 is stored in step S65 with being associated with the IDinformation “30” (refer to FIG. 11). In this case, when one of thecommunication devices 71 and 72 has been selected in step S69 among thecommunication devices 71, 72, 73, 74, and 75, the CPU 22 makes apositive judgment in step S111 (e.g., YES in step S111). When one of thecommunication devices 73, 74, and 75 has been selected in step S69 amongthe communication devices 71, 72, 73, 74, and 75, the CPU 22 makes anegative judgment in step S111 (e.g., NO in step S111). Hereinafter, acommunication device that is selected in step S69 and targeted for theupdate frequency determining processing is referred to as “communicationdevice targeted for processing”.

When the CPU 22 makes a positive judgment in step S111 (e.g., YES instep S111), the CPU 22 designates update frequency information as updatefrequency level “5” (e.g., step S113). That is, when video datatransmitted from the communication device targeted for processing isdisplayed, the update frequency information is designated as updatefrequency level “5”. The designated update frequency level “5” is storedin the RAM 26 with being associated with the ID information of thecommunication device targeted for processing. When the CPU 22 makes anegative judgment in step S111 (e.g., NO in step S111), the CPU 22designates update frequency information as update frequency level “1”(e.g., step S115). The designated update frequency level “1” is storedin the RAM 26 with being associated with the ID information of thecommunication device targeted for processing.

Subsequent to step S115, the CPU 22 judge whether the positioninformation indicates the value “1”, which indicates that a particularcaptured image is being hidden (e.g., step S117). In 5117, the CPU 22accesses the layout information stored in step S65 of FIG. 8 andexecutes the judgment based on the layout information. For example, itis assumed that the layout information transmitted from thecommunication device 30 is stored in step S65 with being associated withthe ID information “30” (refer to FIG. 11). In this case, when one ofthe communication devices 73 and 74 has been selected in step S69 as thecommunication device targeted for processing, the CPU 22 makes apositive judgment in step S117 (e.g., YES in step S117). When thecommunication device 75 is selected as the communication device targetedfor processing, the CPU 22 makes a negative judgment in step S117 (e.g.,NO in step S117).

When the CPU 22 makes a positive judgment in step S117 (e.g., YES instep S117), the CPU 22 raises, by one level, the update frequency levelthat is designated in step S115 for the update frequency information andstored in the RAM 26 (e.g., step S119). Thus, the update frequencyinformation stored in the RAM 26 is designated as update frequency level“2”. When the CPU 22 makes a negative judgment in step S117 (e.g., NO instep S117), or subsequent to step S119, the CPU 22 judges whether aratio of the total video play time, which is associated with the IDinformation of the communication device targeted for processing, to theduration of the ongoing remote conference is greater than or equal to afirst reference value (e.g., step S121). In step S121, the CPU 22accesses the layout information stored in step S65 of FIG. 8 to identifythe total video play time associated with the ID information of thecommunication device targeted for processing. For example, it is assumedthat the layout information transmitted from the communication device 30is stored in step S65 with being associated with the ID information “30”(refer to FIG. 11) and the communication device 73 is selected in stepS69 as the communication device targeted for processing. In this case,the CPU 22 identifies the total video play time of 300.0 sec. The CPU 22determines an elapsed time, which is continuously measured by the timer27 from the start of the particular remote conference upon theestablishment of the connection of the communication device 30 to theparticular remote conference room, as the duration of the ongoing remoteconference. Subsequently, the CPU 22 determines whether a value obtainedby dividing the total video play time by the duration of the ongoingremote conference is greater than or equal to the first reference value.For example, the first reference value may be 0.5 (50%). The firstreference value is stored in the storage device 24 with being associatedwith the programs for executing the update frequency determiningprocessing.

When the ratio of the total video play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is greater than or equal tothe first reference value (e.g., YES in step S121), the CPU 22 raises,by one level, the update frequency level designated for the updatefrequency information stored in the RAM 26 (e.g., step S123). When stepS119 has already been executed, the update frequency information storedin the RAM 26 is designated as the update frequency level “3”. When stepS119 has not yet been executed, the update frequency information storedin the RAM 26 is designated as the update frequency level “2”.

When the ratio of the total video play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is smaller than the firstreference value (e.g., NO in step S121), or subsequent to step S123, theCPU 22 judges whether a ratio of the total audio play time associatedwith the ID information of the communication device targeted forprocessing to the duration of the ongoing remote conference is greaterthan or equal to a second reference value (e.g., step S125). Forexample, the CPU 22 accesses the total audio play time table stored inthe RAM 26 (refer to FIG. 14) to identify the total audio play timeassociated with the ID information of the communication device targetedfor processing. As described above, the timer 27 starts measuring anelapsed time upon establishment of a connection of a particularcommunication device to a particular remote conference room. The CPU 22judges whether a value obtained by dividing the total audio play time bythe duration of the ongoing remote conference is greater than or equalto the second reference value. For example, the second reference valuemay be 0.2 (20%). The second reference value is stored in the storagedevice 24 with being associated with the programs for executing theupdate frequency determining processing.

When the ratio of the total audio play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is greater than or equal tothe second reference value (e.g., YES in step S125), the CPU 22 raises,by one level, the update frequency level designated for the updatefrequency information stored in the RAM 26 (e.g., step S127). When theprocessing of both steps S119 and S123 have already been executed, theupdate frequency information stored in the RAM 26 is designated as theupdate frequency level “4”. When the processing of one of steps S119 andS123 has already been executed, the update frequency information storedin the RAM 26 is designated as the update frequency level “3”.

When the ratio of the total audio play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is smaller than the secondreference value (e.g., NO in step S125), or subsequent to step S127, theCPU 22 judges whether the value representing the bandwidth is greaterthan or equal to a threshold value (e.g., step S129). The bandwidth isstored in the RAM 26 with being associated with the ID information ofthe communication device targeted for processing in step S81 of FIG. 9.For example, the threshold value may be 1 Mbps. The threshold value isstored in the storage device 24 with being associated with the programsfor executing the update frequency determining processing.

In the illustrative embodiment, for reasons of explanation, in thetransferring processing, the CPU 22 judges whether the data received instep S63 of FIG. 8 is layout information. When the CPU 22 makes anegative judgment in step S63 the CPU 22 judges the received data isvideo data in step S75 of FIG. 9. When the CPU 22 makes a positivejudgment in step S75, the CPU 22 obtains the bandwidth in step S81 ofFIG. 9. Therefore, it seems that the update frequency determiningprocessing executed in step S71 of FIG. 8 is executed even when step S81has not been executed yet. Nevertheless, as described above, as aconnection for a particular remote conference is established between theserver 20 and the communication devices 30, 71, 72, 73, 74, and 75,receiving processing (refer to FIG. 6) including, for example, the videotransmitting processing (refer to FIG. 5) is started in eachcommunication device. Subsequently, in step S55 (refer to FIG. 7), thelayout information is transmitted. Therefore, the CPU 22 may obtain thebandwidth, which is a target for comparison, from the RAM 26. Forexample, when there is no bandwidth stored in the RAM 26 at a timing ofexecution of step S129 under special condition, the CPU 22 determinesthe bandwidth as 0 Mbps.

When the CPU 22 judges that the value representing the bandwidth isgreater than or equal to the threshold value (e.g., YES in step S129),the CPU 22 raises, by one level, the update frequency level designatedfor the update frequency information (e.g., step S131). When steps S119,S123, and S127 have already been executed, the update frequencyinformation stored in the RAM 26 is designated as the update frequencylevel “5”. When two of steps S119, S123, and S127 have already beenexecuted, the update frequency information stored in the RAM 26 isdesignated as the update frequency level “4”. When one of steps S119,S123, and S127 has already been executed, the update frequencyinformation stored in the RAM 26 is designated as the update frequencylevel “3”.

When the CPU 22 judges that the value representing the bandwidth issmaller than the threshold value (e.g., NO in step S129), or subsequentto step S113 or S131, the CPU 22 ends the update frequency determiningprocessing and the routine returns to step S73 of FIG. 8.

<Transfer Determining Processing>

Transfer determining processing executed in step S89 (refer to FIG. 9)of the transferring processing in FIGS. 8, 9 and 10 will be describedreferring to FIG. 16. In the description for the transfer determiningprocessing, the video data that contributes to the positive judgment instep S75 of FIG. 9 (e.g., YES in step S75) is referred to as “new videodata”.

Subsequent to starting the transfer determining processing, the CPU 22obtains the update frequency information for ID information (target),which is obtained in step S77 and associated with the ID information ofthe communication device selected in step S85 in the update frequencysetting (refer to FIG. 13) (e.g., step S151). Subsequently, the CPU 22judges whether the update frequency information obtained in step S151indicates that the compressed image in the new video data is determinedas a non-target for transmission (e.g., step S153). The type of thecompressed image in the new video data is identified based on the imagetype information obtained in step S77 of FIG. 9. When the CPU 22 judgesthat the update frequency information indicates that the compressedimage in the new video data is not determined as a non-target fortransmission (e.g., NO in step S153), the CPU 22 judges whether a valuerepresenting a time period elapsed between a timing of the last-timetransmission of an I-frame and a timing of execution of step S151, i.e.,the current time, is greater than a value representing the minimumI-frame transmission interval specified in the update frequencyinformation obtained in step S151 (e.g., step S155).

For example, step S155 is executed as described below. The CPU 22identifies the transmission time of the last-time transmission of anI-frame from the transmission table stored in the RAM 26 in whichvarious information has been stored in step S95 of FIG. 9. The IDinformation obtained in step S77 of FIG. 9 as the ID information of thesender device of the video data, and the ID information of thecommunication device determined in step S85 of FIG. 9 as the IDinformation of the destination of the video data are associated with thelast I-frame of which transmission time has been identified. The CPU 22obtains the current time from the timer 27. In the identification of thetransmission time of the last-time transmission of the I-frame, the typeof the compressed image is determined based on the image typeinformation stored in the transmission table. The CPU 22 obtains anelapsed time based on the last transmission time and the current time todetermine the relationship between the elapsed time and the minimumI-frame transmission interval. When the CPU 22 judges that the valuerepresenting the elapsed time is greater than the value representing theminimum I-frame transmission interval, the CPU 22 makes a positivejudgment in step S155 (e.g., YES in step S155). When the valuerepresenting the elapsed time is smaller than or equal to the valuerepresenting the minimum I-frame transmission interval, the CPU 22 makesa negative judgment in step S155 (e.g., NO in step S155). In otherembodiments, for example, when the value representing the elapsed timeis equal to the value representing the minimum I-frame transmissioninterval, the CPU 22 may make a positive judgment in step S155 (e.g.,YES in step S155).

When the CPU 22 makes a positive judgment in step S155 (e.g., YES instep S155), the CPU 22 stores a determination result indicating“transfer necessary” in the RAM 26 (e.g., step S157). When thecompressed image in the new video data is a compressed image type thatdoes not relate to the minimum I-frame transmission interval, the CPU 22makes a positive judgment in step S155 (e.g., YES in step S155). In theillustrative embodiment, the minimum interval is the minimum I-frametransmission interval. Therefore, the compressed image type that doesnot relate to the minimum I-frame transmission interval may be a P-frameor a B-frame. When the CPU 22 makes a positive judgment in step S153(e.g., YES in step S153) or when the CPU 22 makes a negative judgment instep S155 (e.g., NO in step S155), the CPU 22 stores a determinationresult indicating “transfer unnecessary” in the RAM 26 (e.g., stepS159).

Steps S151 to S159 will be described assuming that the new video data isvideo data received from the communication device 73, the ID informationof the communication device 73 is obtained in step S77 of FIG. 9, andthe communication device 30 is determined as a target to be processed instep S85 of FIG. 9. In the update frequency setting associated with theID information of the communication device 30, the update frequencyinformation for the ID information of each of the communication devices71, 72, 73, 74, and 75 is as shown in FIG. 12. In step S151, the CPU 22obtains the update frequency information for the ID information “73”,associated with the ID information of the communication device 30(sender), based on the update frequency setting (refer to FIG. 12)stored in the RAM 26. According to the update frequency setting depictedin FIG. 12, in this case, the CPU 22 obtains the update frequency level“3”.

When the compressed image in the new video data is I-frame, the CPU 22makes a negative judgment in step S153 (e.g., NO in step S153). When 300msec. has elapsed from the transmission time of the last-timetransmission of an I-frame (e.g., YES in step S155), the CPU 22 stores adetermination result indicating “transfer necessary” in the RAM 26(e.g., step S157). When 300 msec. has not yet elapsed from thetransmission time of the last-time transmission of an I-frame (e.g., NOin step S155), the CPU 22 stores a determination result indicating“transfer unnecessary” in the RAM 26 (e.g., step S159). When thecompressed image in the new video data is one of P-frame and B-frame,the CPU 22 makes a positive judgment in step S153 (e.g., YES in stepS153) and stores a determination result indicating “transferunnecessary” in the RAM 26 (e.g., step S159).

Subsequent to step S157 or S159, the CPU 22 ends the transferdetermining processing, and the routine returns to step S91 of FIG. 9.

Effects of Illustrative Embodiment

According to the illustrative embodiment, one or more effects describedbelow are obtained.

(1) In the server 20, in step S65 (refer to FIG. 8), the layoutinformation received via the communication unit 28 is stored. The layoutinformation is transmitted from one of the communication devices 30, 71,72, 73, 74, and 75 (refer to step S55 of FIG. 7). In step S111, it isjudged whether the position information associated to the ID informationof the communication device indicates the value “0”, which indicatesthat a particular captured image is being displayed. The communicationdevice is selected in step S69 (refer to FIG. 8) and is a target for theupdate frequency determining processing. When a positive judgment ismade in step S111, the update frequency information is designated as thehighest update frequency level “5” (e.g., step S113 refer to). When anegative judgment is made in step S111, the update frequency informationis designated as one of the update frequency levels “1”, “2”, “3”, and“4”, which are lower levels than the update frequency level “5” (referto step S115, S119, S123, S127 or S131).

In the server 20, judgments (e.g., step S153 or S155 of FIG. 16) aremade based on the update frequency information stored as the updatefrequency setting in step S73 in FIG. 8. One of the determination resultindicating “transfer necessary” and the determination result indicating“transfer unnecessary” is stored (e.g., step S157 or S159 of FIG. 16).In the server 20, when the determination result indicates “transfernecessary” (e.g., YES in step S93 of FIG. 9), video data is transmittedto the communication device determined as a target to be processed instep S85 of FIG. 9 (e.g., step S93 in FIG. 9). When the determinationresult indicates “transfer unnecessary” (e.g., NO in step S91 of FIG.9), transmission of video data is not performed.

Therefore, this configuration may enable to determine compressed imagesof particular types as a non-target for transmission, in video datacorresponding to a captured image that is not displayed within the videodisplay region B of the display 38 of the communication device 30. Thatis, this configuration may disable the server 20 to transmit, to thecommunication device 30, video data including the compressed imagesdetermined as a non-target for transmission.

For example, when a particular captured image is hidden in thecommunication device 30, in other words, when the position informationfor the particular captured image indicates one of the values “1” and“2” (e.g., NO in step S111 in FIG. 15), the update frequency informationis designated as one of the update frequency levels “2”, “3”, “4”, and“5” (e.g., step S119, S123, S127 or S131 of FIG. 15) if a positivejudgment is made in at least one of steps S117, S121, S125, and S129 ofthe update frequency determining processing (refer to FIG. 15). When theupdate frequency information is designated as one of the updatefrequency levels “2”, “3”, “4”, and “5”, video data including at leastI-frames is transmitted from the server 20 to the communication device30 even if the video data is video data corresponding to the hiddencaptured image (e.g., FIG. 13, NO in step S153, and step S157 of FIG.16, YES in step S91, and step S93 of FIG. 9). Therefore, for example,when a swiping operation is performed in order to display a hiddencaptured image, the video data including the compressed image that is atarget for transmission is obtained (e.g., YES in step S37 of FIG. 6)and thus the captured image that includes the compressed image andcorresponds to the video data may be displayed in the communicationdevice 30 (e.g., step S43 of FIG. 6). In the remote conference system10, therefore, this configuration may enable to display a hiddencaptured image smoothly in a predetermined communication device (e.g.,the communication device 30) while restricting increase of the amount ofvideo data to be communicated.

(2) The layout information including the position information indicatingthe value “1” is generated for a particular area of the video layoutregion A which is positioned out of the video display region B and ispositioned within the reference distance WC with reference to aparticular reference edge in the direction opposite to the movingdirection (refer to FIG. 2) (e.g., step S55 of FIG. 7). The value “1”that may be position information is higher order value than the value“2” which may be position information that indicates that a particulararea is positioned out of the area corresponding to the referencedistance WC. The position information is transmitted to the server 20.In the server 20, when the position information included in the layoutinformation indicates the value “1” (e.g., YES in step S117 of FIG. 15),the update frequency level designated for the update frequencyinformation is raised by one level (e.g., step S119 of FIG. 15).Therefore, the update frequency information assigned to video datacorresponding to a captured image positioned out of the video displayregion B may be changed from the value “1” to the value “2” withreference to the reference distance WC. Therefore, when the hiddencaptured image is displayed, the captured image may be displayed owingto receipt of I-frames, which are determined as a target fortransmission at the update frequency level “2”. In the illustrativeembodiment, both the moving direction and the direction opposite to themoving direction may be referred to as a direction corresponding to themoving direction. The direction corresponding to the moving directionmay be, for example, the horizontal direction (refer to FIG. 2).

(3) It is judged whether the ratio of the total video play time, whichis associated with the ID information of the communication devicetargeted for processing, to the duration of the ongoing remoteconference is greater than or equal to the first reference value (e.g.,step S121 of FIG. 15). When the ratio is greater than or equal to thefirst reference value (e.g., YES in step S121), the update frequencylevel designated for the update frequency information is raised by onelevel (e.g., step S123). Therefore, the update frequency information forvideo data corresponding to the captured image that is not displayedwithin the video display region B may be changed appropriate to thetotal video play time. For example, when step S117 has not beenexecuted, the update frequency level designated for the update frequencyinformation is changed from the update frequency level “1” to the updatefrequency level “2”. When step S117 has already been executed, theupdate frequency level designated for the update frequency informationis changed from the update frequency level “2” to the update frequencylevel “3”. At the update frequency level “2” or “3”, an I-framedetermined as a target for transmission may enable the captured image tobe displayed. Raising the update frequency level designated for theupdate frequency information from the update frequency level “2” to theupdate frequency level “3” may enable to shorten the minimum I-frametransmission interval (e.g., 1700 msec. shortened), whereby the imageswitching interval of the captured image in display duration of oneframe may be shortened. Thus, reproduction quality may be increased.

(4) It is judged whether the ratio of the total audio play time, whichis associated with the ID information of the communication devicetargeted for processing, to the duration of the ongoing remoteconference is greater than or equal to the second reference value (e.g.,step S125 of FIG. 15). When the ratio is greater than or equal to thesecond reference value (e.g., YES in step S125), the update frequencylevel designated for the update frequency information is raised by onelevel (e.g., step S127). Therefore, the update frequency information forvideo data corresponding to the captured image that is not displayedwithin the video display region B may be changed appropriate to thetotal video play time. For example, when both steps S119 and S123 havenot been executed, the update frequency level designated for the updatefrequency information is changed from the update frequency level “1” tothe update frequency level “2”. When one of step S119 and S123 has beenexecuted, the update frequency level designated for the update frequencyinformation is changed from the update frequency level “2” to the updatefrequency level “3”. When both steps S119 and S123 have been executed,the update frequency level designated for the update frequencyinformation is changed from the update frequency level “3” to the updatefrequency level “4”. At the update frequency level “2” or “3”, anI-frame determined as a target for transmission may enable the capturedimage to be displayed. At the update frequency level “4”, a P-frame isalso determined as a target for transmission. Therefore a P-frame aswell as an I-frame may enable the captured image to be displayed.Raising the update frequency level designated for the update frequencyinformation from the update frequency level “3” to the update frequencylevel “4” may enable to shorten the minimum I-frame transmissioninterval (e.g., 300 msec. shortened) and to obtain P-frames, whereby theimage switching interval of the captured image in display duration ofone frame may be shortened. Thus, reproduction quality may be furtherincreased. The effects obtained due to the raising of the updatefrequency information from the update frequency level “2” to the updatefrequency level “3” are as described above.

<Modifications>

One or more aspects of the disclosure according to the illustrativeembodiment may be modified as described below. One or more aspects ofthe disclosure according to various modifications may be combinedappropriately. Other embodiments that adopt one or more aspectsdescribed below may provide the same effects as the effects provided bythe above-described embodiment.

(1) In the above-described illustrative embodiment, the video layoutregion A is defined by the layout setting that defines that the areasA1, A2, A3, A4, and A5 are arranged along the horizontal direction andthe captured images 381, 382, 383, 384, and 385 are positioned in theareas A1, A2, A3, A4, and A5, respectively (refer to FIG. 2). In otherembodiments, for example, the layout setting may define the video layoutarea such that the areas A1, A2, A3, A4, and A5 are arranged along thevertical direction and the captured images 381, 382, 383, 384, and 385are positioned in the areas A1, A2, A3, A4, and A5, respectively,arranged along the vertical direction. In this case, elementscorresponding to the widths WA and WB, the reference distance WC, andthe distance WD with reference to the horizontal direction may bedefined with reference to the vertical direction. The moving directionmay include an upward direction and downward direction in the verticaldirection. According to the drawing depicted in FIG. 2, the verticaldirection is a direction perpendicular to the horizontal directionindicated by a bi-directional arrow.

(2) In the above-described illustrative embodiment, for the arrangementof the captured images 381, 382, 383, 384, and 385 in accordance withthe layout setting, the areas A1, A2, A3, A4, and A5 are defined in thevideo layout region A (refer to FIG. 3). In other embodiments, forexample, the areas in which the captured images 381, 382, 383, 384, and385 are positioned, respectively, in the video layout region A may beidentified using a coordinate system. In this case, a particular area ofthe video layout region A displayed within the video display region Bmay also be identified using the coordinate system.

(3) In the above-described illustrative embodiment, the positioninformation of the status information includes the value “0” whichindicates a particular captured image is being displayed, and the values“1” and “2”, each of which indicates a particular captured image isbeing hidden. In other embodiments, for example, information that isequal to the position information may be obtained based on informationrelating to the video layout region and information that identifies aposition of a particular area of the video layout region A whether theparticular area is positioned within or out of the video display regionB, for example. The information relating to the video layout region Amay be, for example, information which indicates the arrangement of theplurality of areas in the video layout region A based on the layoutsetting (e.g., M rows by N columns) and information in which theplurality of areas and the ID information of the communication devicesare associated with each other. In the server 20, the CPU 22 may obtainsuch information (e.g., YES in step S63 of FIG. 8). The CPU 22 mayidentify a feature that is equal to the position information, based onthe receive information.

(4) In the above-described illustrative embodiment, in step S13 of theaudio data transmitting processing depicted in FIG. 4, when the valuerepresenting the volume of the audio obtained in step S11 is equal tothe threshold value, a negative judgment is made. In other embodiments,for example, in step S13, a judgment may be made based on whether thevalue representing the volume of the audio obtained in step S11 isgreater than or equal to the threshold value. In this case, when thevalue representing the volume is equal to the threshold value, apositive judgment may be made.

(5) In the above-described illustrative embodiment, in step S81 (referto FIG. 9), the bandwidth is obtained based on the interval of obtainingan I-frame. In other embodiments, for example, the bandwidth may beobtained, for example, on a group-of-pictures (GOP structure) basis. Inthe above-described illustrative embodiment, the description has beenmade in the case where the bandwidth of video data is used. In otherembodiments, for example, instead of or in addition to video data, abandwidth of other data communicated in the remote conference system 10(e.g., audio data, common material data, or control data) may be used.

(6) In the above-described illustrative embodiment, when a negativejudgment is made in step S111 of FIG. 15, the update frequencyinformation is designated as the update frequency level “1” in stepS115. When a negative judgment is made in each of steps S117, S121,S125, and S129, the update frequency level designated for the updatefrequency information is raised by one level. In other embodiments, forexample, the update frequency information may be updated using scores.In this case, for example, the update frequency information and a scoreare associated with each other such that “0 points” indicates the updatefrequency level “1”, “20 points” indicates the update frequency level“2”, “40 points” indicates the update frequency level “3”, “60 points”indicates the update frequency level “4”, and “80 points” indicates theupdate frequency level “5”. The update frequency information may bedesignated in accordance with the score. For example, when a negativejudgment is made in step S111, in step S115, the score is set to 0points and this score is stored in the RAM 26. When a positive judgmentis made in step S117, in step S119, the CPU 22 adds 20 points to thescore. In this case, the score of “20 points” is stored in the RAM 26.When a positive judgment is made in step S121, in step S123, the CPU 22adds “20 points” to the store stored in the RAM 26. When a positivejudgment is made in step S125, in step S127, the CPU 22 adds “20 points”to the score stored in the RAM 26. When a positive judgment is made instep S129, in step S131, the CPU 22 adds “20 points” to the score storedin the RAM 26. Subsequently, the CPU 22 designates the update frequencyinformation in accordance with the final score stored in the RAM 26. Forexample, when the final score stored in the RAM 26 is “60 points”, theCPU 22 designates the update frequency information as the updatefrequency level “4” and determines the update frequency information asthe update frequency level “4” in step S73 of FIG. 8.

In step S121, when the ratio of the total video play time associatedwith the ID information of the communication device targeted forprocessing to the duration of the ongoing remote conference is equal tothe first reference value, a positive judgment is made. In otherembodiments, for example, in step S121, a judgment may be made based onwhether the ratio of the total video play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is greater than the firstreference value. In other words, when the ratio is equal to the firstreference value, a negative judgment may be made in step S121.

In step S125, when the ratio of the total audio play time associatedwith the ID information of the communication device targeted forprocessing to the duration of the ongoing remote conference is equal tothe second reference value, a positive judgment is made. In otherembodiments, for example, in step S125, a judgment may be made based onwhether the ratio of the total audio play time associated with the IDinformation of the communication device targeted for processing to theduration of the ongoing remote conference is greater than the secondreference value. In other words, when the ratio is equal to the secondreference value, a negative judgment may be made in step S125.

In the above-described illustrative embodiment, in step S129, when thevalue representing the bandwidth is equal to the threshold value, apositive judgment is made. In other embodiments, for example, in stepS129, a judgment may be made based on whether the value representing thebandwidth is greater than the threshold value. In other words, when thevalue representing the bandwidth is equal to the threshold value, anegative judgment may be made in step S129.

The order in which judgments in steps S117, S121, S125, and S129 areperformed may be changed. In other embodiments, for example, thejudgment performed in accordance with the condition of step S121 may beperformed at the timing of step S117 executed in the above-describedillustrative embodiment. The judgment performed in accordance with thecondition of step S125 may be performed at the timing of step S121executed in the above-described illustrative embodiment. The judgmentperformed in accordance with the condition of step S129 may be performedat the timing of step S125 executed in the above-described illustrativeembodiment. The judgment performed in accordance with the condition ofstep S117 may be performed at the timing of step S129 executed in theabove-described illustrative embodiment.

(7) In the above-described illustrative embodiment, a judgment is madein step S153 of FIG. 16 based on whether the update frequencyinformation obtained in step S151 indicates the compressed imageincluded in the new video data is determined as a non-target fortransmission. In other embodiments, for example, in step S153, ajudgment may be made based on whether the update frequency informationobtained in step S151 indicates the compressed image included in the newvideo data is determined as a target for transmission. In this case,when the compressed image is determined as a target for transmission,the CPU 22 may make a positive judgment and the routine may proceed tostep S155. When the compressed image is determined as a non-target fortransmission, the CPU 22 may make a negative judgment and the routineproceed to step S159.

(8) In the above-described illustrative embodiment, the server 20transmits video data to each appropriate destination device. The senderdevice of the video data reproduces video from video data obtainedtherein and displays thereon a captured image of itself generatedthrough the reproduction. In other embodiments, for example, the server20 may be configured to transmit the video data to the sender device. Inthis case, the sender device may be configured to display a capturedimage corresponding to the received video data based on the own videodata transmitted from the server 20. In this case, the layoutinformation may include the status information on the captured image ofthe sender device corresponding to video data. The status information isassociated with the ID information of the sender device. In the server20, the CPU 22 may be configured to execute the update frequencydetermining processing (refer to FIG. 15) for each communication deviceidentified by the ID information included in the layout information. Thedetermined update frequency information may be stored in the updatefrequency setting (e.g., step S73 of FIG. 8). The CPU 22 may beconfigured to also execute the transfer determining processing (refer toFIG. 16) for each of the communication devices 30, 71, 72, 73, 74, and75, and one of the determination result indicating “transfer necessary”and the determination result indicating “transfer unnecessary” may bestored (e.g., step S157 or S159). In step S91 of FIG. 9, a determinationis made as to whether video data of the sender device is transmitted tothe sender device itself. When a positive judgment is made in step S91(e.g., YES in step S91), the CPU 22 controls the transmission of thevideo data (e.g., step S93 of FIG. 9).

What is claimed is:
 1. A non-transitory computer-readable medium storingcomputer readable instructions that, when executed by a processor of aserver configured to conduct a remote conference via a network, performa method comprising: a first obtaining operation obtaining video dataand identification information via a communication unit of the serverconnected to the network, the video data being based on an imagecaptured in each of a plurality of communication devices participatingin a remote conference, the video data including a plurality ofinter-frame coded image data and a plurality of intra-frame coded imagedata, and the identification information identifying each of theplurality of communication devices; a second obtaining operationobtaining layout information from a first communication device via thecommunication unit of the server, the layout information including aportion of identification information and status information associatedwith the identification information in a video layout region of thefirst communication device, the portion of the identificationinformation identifying a second communication device different from thefirst communication device, the status information relating to a displaycondition of each of a plurality of captured images in the video layoutregion of the first communication device, each of the plurality ofcaptured images corresponding to the video data of a plurality ofcommunication devices other than the first communication device andincluding the second communication device; a judgment operation judging,based on status information associated with identification informationof the second communication device in the layout information, whether aspecific captured image is included in a video display region, thespecific captured image corresponding to the video data of the secondcommunication device; and a determination operation determining updatefrequency information of the first communication device in response tothe judgment operation judging that the specific captured image ispositioned outside the video display region, the update frequencyinformation indicating a timing at which the video data of the secondcommunication device corresponding to the specific captured imagepositioned outside the video display region is to be updated, and theupdate frequency information setting at least a portion of the pluralityof inter-frame coded image data included in the video data of the secondcommunication device as a non-target for transmission.
 2. Thenon-transitory computer-readable medium according to claim 1, whereinthe method further comprises: a transmission control operationcontrolling transmission of, in response to the judging that thespecific captured image is positioned outside the video display region,the video data of the second communication device in which the portionof the plurality of inter-frame coded image data set as a non-target fortransmission are not included.
 3. The non-transitory computer-readablemedium according to claim 2, wherein the determining operation furthercomprises determining the update frequency information for setting aportion of the plurality of intra-frame coded image data as a non-targetfor transmission for the video data of the second communication devicein response to the judgment operation judging that the captured imagecorresponding to the video data of the second communication ispositioned outside the video display region, and wherein thetransmission control operation controls transmission of the video dataof the second communication device in which the portion of the pluralityof intra-frame coded image data, set as the non-target for transmission,are skipped.
 4. The non-transitory computer-readable medium according toclaim 3, wherein the method further comprises: judging, in response toobtaining the video data of the second communication device and theidentification information by the first obtaining operation, whether aspecific coded image data included in the video data of the secondcommunication device is a non-target for transmission set by the updatefrequency information of the first communication, wherein thetransmission control operation further comprises: causing transmissionof, in response to judging that the specific coded image data is atarget for transmission, the video data of the second communicationdevice including the specific coded image data; and in response tojudging that the specific coded image data is a non-target fortransmission, not performing transmission of the video data of thesecond communication device including the specific coded image data tothe first communication device via the communication unit of the server.5. The non-transitory computer-readable medium according to claim 1,wherein the determining operation further comprises determining all ofthe plurality of inter-frame coded image data included in the video dataof the second communication device as a non-target for transmission. 6.The non-transitory computer-readable medium according to claim 1,wherein the status information, included in the layout information fromthe first communication device, includes position information indicatinga distance to an edge of the video display region in a first direction,the first direction corresponding to a moving direction of the pluralityof captured images in the video layout region, and wherein thedetermining operation further comprises: determining the updatefrequency information to set a portion of the plurality of inter-framecoded image data as a non-target for transmission to be a first intervalfor the video data of the second communication device when the positioninformation included in the status information associated with theidentification information of the second communication device indicatesthat the distance is less than or equal to a reference distance; anddetermining the update frequency information to set a portion of theplurality of inter-frame coded image data as a non-target fortransmission to be a second interval for the video data of the secondcommunication device when the position information included in thestatus information associated with the identification information of thesecond communication device indicates that the distance is greater thanthe reference distance, the second interval being greater than the firstinterval.
 7. The non-transitory computer-readable medium according toclaim 6, wherein the reference distance corresponds to a width of thevideo display region in the moving direction.
 8. The non-transitorycomputer-readable medium according to claim 1, wherein the statusinformation, included in the layout information from the firstcommunication device, includes a total video play time, the total videoplay time indicating a total display duration of each of the pluralityof captured images in the video display region, wherein the determiningoperation further comprises: determining the update frequencyinformation to set a portion of the plurality of inter-frame coded imagedata as a non-target for transmission to be a first interval for thevideo data of the second communication device when the total video playtime included in the status information associated with theidentification information of the second communication device is a firsttotal video play time; and determining the update frequency informationto set a portion of the plurality of inter-frame coded image data as anon-target for transmission to be a second interval for the video dataof the second communication device when the total video play timeincluded in the status information associated with the identificationinformation of the second communication device is a second total videoplay time.
 9. The non-transitory computer-readable medium according toclaim 8, wherein the first total video play time is greater than thesecond total video play time, and wherein the first interval is lessthan the second interval.
 10. The non-transitory computer-readablemedium according to claim 1, further comprises: a third obtainingoperation obtaining audio data and the identification information viathe communication unit of the server, the audio data being based onaudio collected in each of the plurality of communication devicesidentified by the identification information; and a storing operationstoring a total audio play time of the audio data and the identificationinformation in association with each other, the total audio play timeindicating a total play duration of the audio data in a specificcommunication device, wherein the determining operation furthercomprises: determining the update frequency information to set a portionof the plurality of inter-frame coded image data as a non-target fortransmission to be a first interval for the video data of the secondcommunication device when the total audio play time associated with theidentification information of the second communication device is a firsttotal audio play time; and determining the update frequency informationto set a portion of the plurality of inter-frame coded image data as anon-target for transmission to be a second interval for the video dataof the second communication device, when the total audio play timeassociated with the identification information of the secondcommunication device is a second total audio play time.
 11. Thenon-transitory computer-readable medium according to claim 10, whereinthe first total audio play time is greater than the second total audioplay time, and wherein the first interval is less than the secondinterval.
 12. A server configured to conduct a remote conference via anetwork comprising: a communication unit configured to connect to thenetwork; a processor; and a memory storing computer readableinstructions that, when executed by the processor, perform a methodcomprising: a first obtaining operation obtaining video data andidentification information via the communication unit of the serverconnected to the network, the video data being based on an imagecaptured in each of a plurality of communication devices conducting aremote conference, the video data including a plurality of inter-framecoded image data and a plurality of intra-frame coded image data, andthe identification information identifying each of the plurality ofcommunication devices; a second obtaining operation obtaining layoutinformation from a first communication device via the communicationunit, the layout information including at least a portion of theidentification information and status information associated with theidentification information in a video layout region of the firstcommunication device, the portion of the identification informationidentifying a portion of the plurality of communication devicesincluding a second communication device different from the firstcommunication device, the status information relating to a displaycondition of each of a plurality of captured images in the video layoutregion of the first communication device, each of the plurality ofcaptured images corresponding to the video data of the portion of theplurality of communication devices; a judgment operation judging, basedon status information associated with identification information of thesecond communication device in the layout information, whether aspecific captured image is included in a video display region of thefirst communication device, the specific captured image corresponding tothe video data of the second communication device; and a determinationoperation determining update frequency information of the firstcommunication device in response to judging that the specific capturedimage is positioned outside the video display region, the updatefrequency information indicating a timing at which the video data of thesecond communication device corresponding to the specific captured imagepositioned outside the video display region is to be updated, and theupdate frequency information setting at least a portion of the pluralityof inter-frame coded image data included in the video data of the secondcommunication device as a non-target for transmission.
 13. The serveraccording to claim 12, wherein the method further comprises: atransmission control operation controlling transmission of, in responseto judging that that the specific captured image is positioned outsidethe video display region, the video data of the second communicationdevice in which the portion of the plurality of inter-frame coded imagedata set as a non-target for transmission are not included.
 14. Anon-transitory computer-readable medium storing computer readableinstructions that, when executed by a processor of a server configuredto conduct a remote conference via a network, perform a methodcomprising: a first obtaining operation obtaining video data andidentification information via a communication unit of the serverconnected to the network, the video data being based on an imagecaptured in each of a plurality of communication devices participatingin a remote conference, the video data including a plurality ofinter-frame coded image data and a plurality of intra-frame coded imagedata, and the identification information identifying each of theplurality of communication devices; a second obtaining operationobtaining layout information and identification information from each ofthe plurality of communication devices via the communication unit of theserver, the layout information including at least a portion of theidentification information and status information associated with theidentification information in a video layout region of a specificcommunication device, the status information relating to a displaycondition of each of a plurality of captured images in a video layoutregion of the specific communication device, each of the plurality ofcaptured images corresponding to the video data of a portion of theplurality of communication devices corresponding to the portion of theidentification information, the specific communication device being oneof the plurality of communication devices; a storing operation storing,in a storage device of the server, the layout information and theidentification information in association with each other; a judgmentoperation judging, for the layout information associated with theidentification information other than identification information of thespecific communication device, whether a specific captured image isincluded in a video display region of each of the plurality ofcommunication devices other than the specific communication device basedon status information associated with the identification information ofthe specific communication device in the layout information associatedwith the identification information other than the identificationinformation of the specific communication device; and a determinationoperation determining update frequency information in response to thejudgment operation judging that the specific captured image ispositioned outside the video display region, the update frequencyinformation indicating a timing at which the video data of the specificcommunication device corresponding to the specific captured imagepositioned outside the video display region is to be updated, and theupdate frequency information setting at least a portion of the pluralityof inter-frame coded image data included in the video data of thespecific communication device as a non-target for transmission.
 15. Thenon-transitory computer-readable medium according to claim 14, whereinthe method further comprises: a transmission control operationcontrolling transmission of, in response to judging that the specificcaptured image is positioned outside the video display region, the videodata of the specific communication device to a communication deviceidentified by identification information associated with layoutinformation which is a subject to the judgment operation, the portion ofthe plurality of inter-frame coded image data set as a non-target fortransmission are not included in the video data of the specificcommunication device to be transmitted.